Regular expression to search for Gadaffi | Hacker News Reader

Regular expression to search for Gadaffi

stackoverflow.com

74 pointsmarcog125 comments

Easy... (Qadaffi|Khadafy|Qadafi|...)... it's self-documented, maintainable, and assuming your regexp engine is actually a regexp engine, will compile to the same DFA that the obfuscated garbage in the SO solutions will compile to.

Writing compact regular expressions is like using short variable names to speed up a program. It only helps if your compiler is brain-dead.

> assuming your regexp engine is actually a regexp engine

There is no guarantee that two regex that match the same corpus will compile to the same DFA or even compile to a DFA at all. Do not confuse regex with a regular expression.

The answer isn't to turn against any of these variation of regex by deriding them as "compact" but instead test to see if this is a bottle neck, and try out a non-self documenting variation if it is.

You are right, I should be more precise with my terminology. By "regexp" I meant "regular expression". Regardless, neither term implies implementation as a DFA, however regular expressions, and the subset of regexps which are equivalent to them, can and should be compiled to an optimized DFA.

A cartesian multiplication is not necessarily more readable, if it takes exponentially more code.

A bit more verbosity over [QKG]add?... could be nice. For example, Haskell's parsec looks like:

do oneOf "QKG" string "ad" optional (char 'd') ..

This also has the nice benefit of generating nice errors when it fails ("Expected 'ad', found ...").

Isn't it strange that there is no official transcription for Arab names?

Chinese, Japanese and Korean used to have a transcription by language (i.e. a transcription for the English language, another one for the french), but that got standardized, and now there is only one transcription for each word (except of course names that became really common: Peking in English, Pékin in French, Beijing in the now official pinyin transcription). Is it because the Arab language is separated in so many countries without a centralised system to "manage" the language?

Not sure what the complete answer is, but part of it is that spoken Arabic varies quite between Arab countries. Do you try to match the pronunciation within that country or the way it would be pronounced in standard Arabic?

I had similar thoughts when I talked to people from Israel. My understanding is now that they can more or less define freely how they are mapping their official hebrew name to latin characters (passport contains both versions).

Similar problem imo and that in spite of being the country "in charge of hebrew".

In print, Arabic uses MSA (Modern Standard Arabic) [1]. Much like Latin (now), it's simply a written language. Spoken there are many dialects. There is however a vast difference between the dialects and MSA. I read one description that said French is to Latin what Colloquial Jordanian Arabic is to MSA. Not knowing Arabic at all I can't speak to the truth of that claim.

As one could guess, illiteracy is a bigger problem in Arabic countries because of this.

Now as for transcription of Arabic characters to English, it is difficult [2], partly because the concepts don't necessarily translate and also because pronunciations vary between Arabic dialects.

Also, the fact that Arabic is spoken in many countries means there is no central authority, much like English.

[1]: http://en.wikipedia.org/wiki/Modern_Standard_Arabic

[2]: http://en.wikipedia.org/wiki/Romanization_of_Arabic

> Much like Latin (now), it's simply a written language.

Really? Suppose a Moroccan and an Iraqi are talking. Suppose also that these are educated, literate people who understand MSA. Wouldn't they talk in MSA, or at least something halfway between their local dialects and MSA? (I'm assuming here that Iraqi and Moroccan Arabic are not entirely mutually understandable -- is this correct?)

They wouldn't speak in MSA. Except in contrived settings, no two people would ever exclusively converse in MSA.

In your example, they would meet one another halfway as you said. One would speak slower and substitute MSA words for colloquial ones here and there.

Also, MSA is not like Latin in that it's exclusively written. It's spoken in various official scenarios, e.g. news broadcasts and speeches. I'm unsure what the future of it will be. I used to be pessimistic, but now with the shared widespread access to the Internet (blogs, YouTube), the Arab world will need a lingua franca again.

>I'm assuming here that Iraqi and Moroccan Arabic are not entirely mutually understandable -- is this correct?

Moroccan Arabic is very hard to understand for Iraqi people. In fact, if you're an Arab from the Asian part of the Middle east, you may need subtitles to watch a local Moroccan movie.

>Suppose a Moroccan and an Iraqi are talking. Suppose also that these are educated, literate people who understand MSA. Wouldn't they talk in MSA, or at least something halfway between their local dialects and MSA?

What usually happens between Arabs from close-dialects countries is that they speak their local dialects and use the MSA to express the different, unfamiliar words.

But between very far, quite different dialects (e.g. Moroccan and Iraqi), they often speak MSA, entirely. (Native speaker here)

No one country to 'officially' manage it is likely a big factor, as is the fact that Arabic has a sort of diglossia between Modern Standard Arabic and the "dialects" spoken in each country. For names in particular, they are sometimes transcribed in a more phonetic way, trying to capture how people actually pronounce them in, say, Libya or Morocco or Egypt; and other times are transcribed in a more structural/etymological way based on the spelling, following one of the schemes for transcribing the Arabic alphabet into English, which are usually based on Modern Standard Arabic.

Greek has a version of the same variation, despite not having modern-day diglossia and being mostly associated with one country, because of the ancient/modern relationship. For example, the large peninsula in southern Greece, Χαλκιδική, is spelled: Halkidiki, Halkithiki, Halkidhiki, Chalcidice, Chalcidiki, Khalkidhiki, Chalkidike, etc. Some of the transliterations attempt to capture how it's pronounced in modern Greek (Halkidiki is close, but Khalkidhiki captures some of the nuance better if you know what the 'kh' and 'dh' sounds are supposed to be), others try to transcribe the Greek alphabet using one of the standard schemes for either ancient or modern Greek (Chalcidice is the classic transliteration), and others do some mixture.

People tend to have strong opinions as well; classicists and etymologists don't like the obscuring of spelling that the phonetic transcriptions produce, but Greeks wince at people pronouncing Halkidiki as if it started with the English word "chalk" and ended with "dice". I imagine many Arabs similarly favor the more phonetic spellings for their names.

I totally agree. I'm a native speaker and my Arabic name (ضياء) has no definite, accurate transcription in English for another reason.

Arabic language has many letters and sounds western languages don't have equivalents of and therefore foreigners are completely unable no pronounce.

Another point is that vowels have different characteristics in different languages; height of the tongue, backness, roundedness, nasalization, ..etc, make it almost impossible to write the correct equivalent for some Arabic vowels in English. For example, we have a word (هيل; means cardamom) whose vowel length is between hail and hell that I can't think of any accurate way to transcript in English.

I don't think vowel differences are sufficient to deem a language impossible to romanize. Japanese has been romanized effectively for decades despite the fact that Japanese vowels are quite different from English ones.

I think this comes down to two competing approaches in transliteration:

1. Transcribe the original non-roman text into something that a native English speaker would read correctly without help. This never works well due to vowel differences and accents, but nevertheless it works well enough that it's very popular. An example of this failing horribly:

Hiroshima: often pronounced Hih-RO-shih-mah. Actually pronounced Hee-roh-shee-mah (nevermind the Japanese R).

2. Romanization without intent to be pronounceable. Nihon-Shiki romanization seems to use this approach: it's unreadable by English speakers, but unlike other romanization schemes, provides a one-to-one mapping to the original kana, i.e. it loses no information.

I think it's important to note as well that "roman letters" != "English"; Nihon-Shiki is an example of this, the creation of roman letters intended to be pronounced by non-English speakers.

Chinese has only one transcription for each word? I guess you've never been to Taiwan. It only seems that way for Chinese because PRC is so much bigger. Taiwan uses a different romanization system.

I imagine the difficulty for Arabic would lie in the fact that there is no major player like China in the Arabic world.

Actually, if you look at street signs in Taiwan you'll see that the use multiple Romanization systems, often in the same city!

I think it was in Kaohsiung that I saw the same character Romanized two different ways on two different streets.

I'm not sure, but I think they're using some combination of Pinyin, Wade-Giles, and possibly something else.

Even worse, I think some of the Romanizations used in Taiwan are just made up on the spot. My wife's name should be written "Hui Ling" in Pinyin, but for some reason her English teacher in Taiwan (who was Taiwanese) told her to write it as "Huey-Ling". So now we get phone calls asking for "Huey" all the time.

I thought that the ROC (Taiwan) government recently adopted pinyin as the official transcription of Chinese characters. All the other countries with Chinese as an official language have done so (Singapore, Malaysia, Indonesia), so it seemed logical.

Do the local governments of Taiwan administrative division apply other transcription?

(G310|K310|Q310)

http://en.wikipedia.org/wiki/Soundex

I wouldn't expect too many false positives.

I just came here to post much the same thing. In my answer I combined the metaphone results (also limited) to remove false positives... I don't think a Mr. Godby would be too pleased to have airstrikes on his property due to using too broad a matching system :)

http://stackoverflow.com/questions/5365283/regular-expressio...

That's actually a pretty good rule. But I checked the list on Stack Overflow and we might also need to include: Kazzafi - K210 Qadhdhafi - Q331

K210 a bit more problematic, since it matches names like 'KOSOFF'.

Me: "Kazzafi"

Google: Did you mean "Gaddafi"?

Hmm, I think I have a potential solution...

while the regexp would find Ghaddafi in all latin script languages even if it's not understood, then let me suggest my Russian as well :)

Каддафи

btw, it's always written only this way, to the opposite of the Osama bin Laden situation, when some emigrant sources name him as Осама бин Ладен instead of the correct Усама бен Ладен.

Also see http://upload.wikimedia.org/math/6/1/f/61f34aa25871e9546b6a1...

From the SO page: "I just thought that if the arabic transcription says Qaḏḏāfī, the regex should check for Qaddafi too."

Would any of these searches find an article that solely uses a phonetic spelling with diacritical marks as opposed to a "standard" alpha only transliteration?

I guess it depends on the engine used - in which case the question is do any regex engines do such automated conversion?

There are over 32 accepted ways of spelling Mu'ammar al-Qadhafi. Here's a dance song to help you remember just a few of them: http://www.buzzfeed.com/danilic/gaddafi-the-duck-sauce-remix...

Reply on news.ycombinator.com