Search This Blog

Tuesday, December 04, 2007

How Do Arabization Works? The Idea behind Arabization

Peace be upon you,



as requested from many guys I am writing this topic, some people was asking how the Arabization works?
here is a brief answer for this question, I will sort it be stages,
First: you have to ensure that the system has a free font type support
why is it so important?
simply, the free type gives you a very good ability which is the system will draw the character for you, you don't have to think how the fonts will look like, how will u draw it all of this question will be disappear with simple idea, it is simple but powerful
What if the system doesn't support Free Type?
you will have to do it your self, you will have to draw the character by your self, for example, you can put the character in BMP or JPG, and do some graphics engine that put characters together and make it look like one sentence
Second: Bi Di, this stage is very important and take a lot of time to be done, if you are going to do it by your self,
fortunately that there is library and it is open source, it is called "FriBiDi"
here is links for this GREAT project
http://sourceforge.net/projects/fribidi

http://fribidi.freedesktop.org/wiki/
this library will help you to rearrange the Arabic Characters, because Arabic from right to left, the whole words will be reversed, so you will have to fix it, also there is many consideration that this library solve, like what if there is English word in the middle of the statement, then it will be MESS, this library will solve the problem

there is one more thing, when I was implementing the algorithm I face a very nice problem, the whole word length, when you fix the string you make like, the first character will be the last and vise versa, when you put this statement in Text Box or any object to display, you will have this problem which is , the lines will be reversed :D
the problem can be solved by detecting the character width, and display area width, so you only put what is enough to display per line, this will help to avoid this problem

Last : Shaping, shaping is very nice and it is totally take no time to do it, simple you will detect the character location ( Start, Middle, End or Standalone ) because Arabic character has 4 shapes, you have to do this part, after that you will find the Arabic string looks OK, and ready to print

How did I do that with Android?

simply when I was watching the Introduction video, I notice that the guy said "Free Type Support"
I smiled then I open the Emulator, and I started looking for the font folder,
as I don't know how to register new font in Android, I replaced the Original font
the cool part comes when I noticed that they have already done the Bi Di support
at the moment I released that all it needs is just Reshape the character the statement
I did this part and guess what it WORKS

there is other many consideration about the Arabization, if you face some please don't hesitate to ask




I hope that this topic was informative for you :)



Thanks for your time

Ahmed Essam

24 comments:

OHDev said...

Asalamo Alaykom ...

Thank you ahmed ...
I went through similar Issues because Arabic is supported yes, but not always and sometimes with Limitations.

some software vendors prefer not to support arabic at all.

anyway most of the sources i found on the Internet talk about other right-To-Left languages such as Hebrew, Asian Languages.
Minor ones talk about Arabic.

This article is most usefull for those who deal with Embedded devices, mobile Development and Linux, UNIX Development.

thanks again!

Ahmed Essam Naiem (Neo_4583) said...

Peace be upon you

this one of the reasons that push me to write this Article, actually I will do more for it, I have some code that I have wrote for such thing, but this stuff need some time for investigation,

anyways, I am ready to be a RESOURCE for any one who need any support for the Arabic Issue

Mohammad Tayseer said...

Nice, but I will guess they will shape the text before final release.

Ahmed Essam Naiem (Neo_4583) said...

Salamo 3lykom

Are you talking about Android?

as I think, they like to open the market for 3rd party companies, u know that windows mobile doesn't has Arabic support, Imaginet is the biggest company for the Arabic support for pocket pc(BTW I was working there but on something else not in the Arabic Support), also the issue is not about the Arabic support it self, you know that loading fonts comsume some memory, they will not add the Arabic for few reasons, first if they are going to support arabic then they should support all languages, do u know how much the size of the font file that has all languages it's about 18 Mega Bytes, it is too much for small machines like mobile, I tried once to put it the device get hang, I think they will do like Microsoft, many copies for the Windows mobile localized depending on the region, there is no envy about the Arabic if someone gonna think like that, Microsoft invest on what is getting real benefit right now there is no benefit from adding arabic support by default, if you think about the capability, Microsoft can easily add the arabic support.
by the way , I have comment about the Arabic support on Windows, while I am doing some experiments on it I found that the Arbaic support that is provided by Imaginet is more efficient than the defualt Arabic Support that on normal Windows, until now I Don't get deeper to know why, but I think that guys in Imaginet have done a lot of work to enhance it over the time, it they did it good,
I hope to hear back from you Mohamed

thanks for your time :)

Happy said...

Salam alikom
very nice article Ahmed, very informative, I'm trying to implement the same mechanism on Android but still some problems, and would like to ask if you can share your code to integrate Arabic language on Android, it will help many people for sure.
Jazakom allaho Kairan

Ahmed Essam said...

Peace be upon you "Happy"

Before anything to say here is link for what I have got after working for while under Android "I managed to show Arabic Strings on the screen"
http://www.ahmed-essam.com/2007/11/applicability-for-arabizing-android.html

After that I have open source project for Devices Arabic support
and I have done some work for Windows mobile and Android
please review this link

http://www.ahmed-essam.com/2008/03/anroid-with-arabic-free-class.html

here is the project link
https://sourceforge.net/projects/wm-arabic/

I hope that you find the blog informative.

Happy said...

Salam alikom
Jazakom allaho kairan for your help, I used your class but I noticed that you didn't use freeBidi as you descibed in your post, it's another solution? also there is some problem with the class to render correctly the first and the last character.
anyway Thanks a lot and allah al mosta3an

Ahmed Essam said...

Paece be upon you

As I explained in Android post, Android team has already implement 2 important things, first the engine of True Type Font, second the Bidi, so the only thing that is needed in Android was the reshaping class which I provide in the open source project, the class is never been tested I think that it is good to be tested :) thanks a lot for that, I hope that you fix the reshape problem in the class in check it in back :) it will be great if you do so, also there is one problem, the class doesn't solve the Lam Alf, I hope that you success in what you are doing :)

ALLAH bless you.

BR
Ahmed Essam

Happy said...

Jazakom allaho kairan brother Issam for your support.

just one more question, in your post http://www.ahmed-essam.com/2007/11/applicability-for-arabizing-android.html you used Freebidi to get to display of Fig3??
and if so, you didnt release it yet?

I will try to improve your class, and just you know, I'm working with my friend on a Quran application for Android that let user read and search verses.
Allaho al most3an
Salam alikom

Ahmed Essam said...

Peace be upon you

First of all.

Andorid has the BiDi, you don't have to do the BiDi

the only few things you need with Android, is to install fonts, which is done by copying some font files in the system\font folder.
the last thing you need is to reshape the Arabic character from its current state to known like middle shape or last and so on.

Amr Ismail Gawish said...

Salamo 3alikom

Thanks for your effort in this great job, You've really done great things.

Your job are highly appreciated.

Thanks for your effort

Amr Ismail Gawish said...

Salamo Alikom...
Sorry but one last question...

How did you replace the font in the Android with the font of your choice...

Thanks in Advanced

Ahmed Essam said...

I have replied in the Android post

rashid a said...

asalam alaykoum...
hello ahmad..
firstly i would like to thank u for ur effort and great work.... we really need to see ur work on the android market if u want i would donate and i will take the charges for publishig ur app on the market for all of the arab users on the android such as me and a lot more..
plz bro help all the arab users who don't know how to do it ...

thank u ur brother Rashid Al-Kuwari

dov said...

Glad you like fribidi.

Regarding Android, I'm curious if it has also resolved other BiDi issues like hit detection, split cursor, selections, automatic base direction etc. After all, full BiDi support is a lot more than just the BiDi algorithm.

Regards,
Dov Grobgeld - Initial author of fribidi

Ahmed Essam said...

I know the point that you are talking about but I am not sure if it is done in the Android or not.

currently I am checking out a version of the Android source code to look for this part.

there is too many considerations shold be taken while developing this thing

- Direction of the statements in case that statement starts with "RTL" language letter
- Selection, in manar of "correct selection to the targeted character" not dummy turned over selection (considering RTL on the selection)
- Editing the rich edit content correctly in like browsers and document editor like office and open office (if it exist for Android)

I think the experince is little bit different than windows mobile if you tried it.
Windows mobile is closed and somehow it is very stupid to do on top of stack that should be done in the core of the system.

What I have noticed that Android fixes the line width correctly, which means initally we have something working, I didn't check their freetype library but I think it is well done, because of that Google cares about performance that why I think it will be done in good manar.

I will try to write more about what I have find out with Android

Thanks for your time.
BR
Ahmed Essam

Anonymous said...

check this out: http://blog.amr-gawish.com/39/arabic-language-in-android/

G_man said...

hey,

Thanks ahmed for everything

I have an android phone and have access to all system files

I have changed the font and it reads arabic

I downloaded the reshaper but I have no idea how to integrate with the system

Can you help me with this part?

I am even able to write arabic now using a virtual arabic keyboard.

I would really like to add the reshaper to my phone.

Thank you and I hope my message does not annoy you.

May Allah bless you

Ahmed Essam said...

Peace be upon you

you seem to be not a software developer.

The class can be used on local application only, it means it will work with developers who uses this class, like what happened with the Arabic SMS Reader and Star trasnlator both uses my class

I hope taht I can help in Enabling Arbaic all over the system.

Thanks for your time.
BR
Ahmed Essam

Jaffar said...

this is magnificent job, I'm just wondering if it's possible to implement this class in rooted Nooks e-reader which are based on Android OS, using Arabic fonts is no problem since we can import or specify the font location on the SD card, the only problem is reshaping of the letters , I'm not a software developer , and I would like to be able to read Arabic books on the nook

Ahmed Essam said...

Peace be upon you Jaffar, I am glade that you liked the post.

Actually the thing about the Nook is it has dual screen, and I am not sure about Android and how it renders the text on such device unfortunately I am resident at Egypt, such device aren't available here, but u still can try to put the fonts on the font folder and try to have some fun with it I think it will need some tricks to make Arabic running Ok on the device, I promise if I got my hands on Nook I will do what I can do with it and let you know :)

Thanks for your time.

moamen said...

how can i use arabicreshape class in listview

or
how can i let listview display arabic

ASSADesign said...

hi sir i have android 2.1 how to support arabic language to my sms and browser.
my phone only comes with russian and english only please i need your help for this
thanks for your coperation

Prashant_mad said...

Hello sir how to show server Response
مهارات القراءة (1) in android. it takes whole characters in brackets.
Plz help very Important.

Prashant.cse2007@gmail.com

waiting for your reply.

Thank you

There was an error in this gadget