Wikifunctions
wikimediafoundation.org
wikimediafoundation.org
cf. https://www.wikifunctions.org/wiki/Wikifunctions:About
> Wikifunctions is a Wikimedia project for everyone to collaboratively create and maintain a library of code functions to support the Wikimedia projects and beyond, for everyone to call and re-use in the world's natural and programming languages.
It's a support project of the Abstract Wikipedia initiative to model facts with data not specific to any spoken or written language, building on and supporting things like Wikidata.
Welcome to Wikifunctions - https://news.ycombinator.com/item?id=36927695 - July 2023 (163 comments)
And a much more interesting example which helps illustrate why Wikipedia built this: "genitive case of Bangla word in Python": https://www.wikifunctions.org/view/en/Z10594
function Z10070( Z10070K1, Z10070K2 ) {
return Z10070K1.includes(Z10070K2);
}
> In particular, according to Keet, Grammatical Framework notably lacks the flexibility to handle certain aspects of Niger-Congo B languages’ morphology. The traditional answer to this kind of critique would be to say, “Let’s just organize an effort to work on Grammatical Framework.” The design philosophy behind Abstract Wikipedia is to make as few assumptions about a contributor’s facility with English and programming experience as possible. Organized efforts around Grammatical Framework (or otherwise) are not a bad idea at all. They may well solve some problems for some languages. However, the contributor pools for under-resourced languages are already small. Demanding expertise with specific natural languages like English and also specific programming paradigms contracts those pools still further.
and
> A solution designed by a small group of Westerners is likely to produce a system that replicates the trends of an imperialist English-focused Western-thinking industry. Existing tools tell a one-voice story; they are built by the same people (in a socio-cultural sense) and produce the same outcome, which is crafted to the needs of their creators (or by the limits of their understanding). Tools are then used to build tools, which will be again used to build more tools; step by step, every level of architectural decision-making limits more and more the space that can be benefitted by these efforts.
and
> The evaluation acknowledges little to no value to the effort that Wikifunctions is investing in being multilingual in terms of natural languages, even for implementations. It is unclear as to how a solution that is built on top of (the existing Lua-based Scribunto scripting system) would succeed in being multilingual, and not just be an English project which the other languages may be invited to use, if they understand it, but would be mostly blocked from contributing to.
This of course only works if all functions, and all parameters, etc have had labels submitted for your language.
They indicate wanting to cater to cultures that might not use a base 10 representation of numbers, so would presumably autotranforms numeric literals for people from those cultures.
They are worried about programming languages being englished based, but seem to think that similar autotranslating of keywords is a good enough approach, so that is why they NEED to support multiple languages, so that they might hypothetically support some language that supports base 60 math and keywords based on [insert language here].
Plenty of room for abuse here, as it would be really easy to edit functions being used for text generation of less popular languages to output offensive text, for things not tested by the test cases. And nobody except people who know that language will be able to tell for sure that this is not a legit contribution.
Then of course people will by trying to code sandbox and then container breaks, cryptominers, etc. I expect a giant game of whack-a-mole between the project and the admins.
On that note even ignoring the abuse, or people trying to utilize the service for free compute resources, I can see the costs of this service becoming substantial really quick. Just from things like legitimate contributions of test cases, and new language implementations, and running the test cases again all the different implementations.
This is already the case with wikipedia, there was the recent story that most of the Scots wikipedia was written by someone who does not know the language, in the style of "English, as she is written".
WTF. Virtue-signaling identity politics has no place in computer science.
In my opinion this project is about 20 layers too abstract to actually produce working and useful software. But it's not virtue signaling if they're actually trying to do the thing, is it? Virtue signaling is when you act like you're doing something virtuous and then don't.
If they're trying to create a programming syntax that maps to every known human language, good on them. They'll probably fail, but it's not really a stupid concept, especially when your organization's purpose is to spread knowledge to the whole world.
If one belives avoiding well-known tools and (english-based) programming languages is likely to actually make it easier for contributors using those languages to contribute to Wikipedia in those languages, then it's real. If one believes that this is not actually going to result in more usable tools, and the intention of avoiding tools from English-speaking Westerners is signalling rather than actually getting better results, then it's virtue signalling.
I'd think us, or any foundation for that matter, should be able to lay out an agenda before outcomes are achieved, how (un)realistic they may be. A dot on the horizon to aim for, not necessarily achievable but a heading at least.
If you're just saying it for the upvotes, yes. If you're actually trying to get people to shift their efforts towards more effective things (such as translation support libraries that don't insist on avoiding English at every level), then no, it's legitimate effort.
> I'd think us, or any foundation for that matter, should be able to lay out an agenda before outcomes are achieved, how (un)realistic they may be. A dot on the horizon to aim for, not necessarily achievable but a heading at least.
An aim is fine. It's a question of sincerity.
Unless you equate "global" with "imperialist", in which case your statement is true by definition. But then everything that is done on a global scale is necessarily imperialist, and then one of the two terms does not seem useful anymore.
Equity can be global.
That would require the equation to be wrong.
For example, if the same sentence was formulated as something like "expanding the contributor community by including people knowledgeable and proficient in languages not commonly spoken in the West and soliciting feedback from such contributors would greatly expand the appeal of the resulting solution and make it more widely applicable" - wouldn't that sound more nice to people?
> A solution designed by a small group of people is likely to produce a system that replicates trends specific to this group and are not universal enough for our need.
The fact that I couldn't type my name for decades in most IT systems because of the charsets used is not "virtue-signaling" to complain about, thank you.
> with needing to support the vagaries of every foreign dialect in the world
Then don't impact the Roman software on people outside Rome. Basically the definition of imperialism.
Edit:
> I think the Internet was a better place before Eternal September, when it was the exclusive precinct of relatively affluent white male nerds. (...) I think elitism is right.
Ah, I see this from you in another thread. So you're basically against other cultures and groups being present online. No wonder you want everyone to conform to your definition of how things should be. Too bad the ship has sailed, we're here.
You (or others you depend on) are choosing to use "Roman" software.
> I'm trying to book an airplane ticket in my country. But it's not possible to write my name, even though my name contains letters found in our alphabet.
It is possible to write your name in a way that lets the airline and airports identify you, which is what matters. It might not be possible to write it exactly the way you'd like but that's not causing any real issues except making you upset.
It's also pretty ironic to complain about having to deal with other cultures for (presumably) international travel.
> Then don't impact the Roman software on people outside Rome. Basically the definition of imperialism.
"Anglocentric" software isn't forced on anyone. You can campaing for your country to disregard all English writings and re-discover all that knowledge from first principles instead if you'd prefer. But that would make you much worse off, wouldn't it?
English-first computing dominates precisely because not having to deal with localization (or at least not initially) lets people focus on what actually matters to make work. Don't like it? Write your own software instead of complaining that relying other people's work means you sometimes have to adapt a bit.
> So you're basically against other cultures and groups being present online.
Plenty of people from other cultures (including me) have no problem with learning English to interact with the world and tap into the vast pool of English knowledge and tools. Language != Culture.
> Too bad the ship has sailed, we're here.
So is English as the lingua franca of the computing age.
Computer programming uses a small subset of English in code and the industry nearly always uses the American socio-cultural kind of English.
English is the fullest language we have, with all the infamy of pilfered concepts from the European continent.
The language/natural language distinction is reductionist, halving all language, when in reality it is a structure that cannot be halved.
He recommends reducing languages down to their most basic elements. This is a mistake that doesn't go anywhere. You end up with a pile of sticks.
All his crictisms of English (one voice, limited understanding, ect) can be leveled at any living, breathing human.
I wonder where the experiment goes. His criticisms of English should be better.
When did we become Westerners and started to think in the same way?
I recognised a trend for such expressions in the news, too, in the last few years. It's misleading and creates a "we against them" mentality.
The West is essentially all the world's nations whose cultures are derived from or greatly influenced by Latin speakers and Judeo-Christian values, which traces its influence back to Ancient Rome.
It's a very valuable and important thing to recognize, because things that people in the US and Europe often implicitly assume are human universals, aren't. They're only universal across the West, and not found in e.g. China or India.
So it's not creating division. It's recognizing difference that already exists, but that not everyone is always already aware of, because we tend to exist in our own little bubbles.
Having a lingua franca for computing is a good thing. All these efforts to localize computing are nothing but a giant waste of time.
this is also related to how 2 hour long movies will diminish into a niche... and movies of the future will be more like [0]
this observation is on the same trend as operas; which way back would last all evening (over 3 hours was typical)
this rant gave me a proper laught. thank you.
def Z10591(Z10591K1, Z10591K2):
word, is_ra_plural = Z10591K1, Z10591K2Just as each Wikipedia article is about one "thing", each Wikifunction performs one job. Once you know a Wikifunction works reliably, you know every other Wikifunction which depends on it can theoretically trust its output.
This project seems really exciting, especially in the context of AI. Very cool and I hope to see it flourish.
Stares at C/C++
It's like saying you have an "empty cup of water".
So, but wait... Is the genitive "of water" functioning like an assertion? "All of the contents of the cup are water."
And wouldn't that assertion, in the case where the cup has no contents, be a vacuous truth?[0]
In other words, a cup of water, if it actually has water in it, cannot also be a cup of magma. But there's nothing stopping an cup with no contents at all being "of" both.
It's a bit like that with pointers in C: a char* is not a int* but null pointers are convertible to any pointer type.
For example, in Jewish religious families, there are often two sets of kitchen utensils - for meat products and for dairy/other products, since mixing those makes the food unfit for consumption by a religious person. In this situation, if you said "I have an empty meat pot" and somebody answered you "No, I need an empty milk pot, give me another one!" - it would make a total sense. Even though both pots are empty, they are not the same kind of pot for their users.
Similarly, if a string is a sequence of certain objects (characters, bytes, runes, graphemes, whatever you please) then empty sequence of such objects is not the same as an empty sequence of other objects - you could append a character to it, but you can't append a database connection to it, for example, and expect something sane to happen.
So yes, "empty cup of water" is exactly what it is, because in most programming languages, empty cups are not all alike. In some languages, there are sequence types that are agnostic towards their elements, and then an empty sequence of that kind wouldn't be a sequence of anything - it'd be just a separate entity. But strings rarely are implemented this way, for many practical reasons.
It's like a bad student that when the teacher says "assume the train departs from the station at 9 am", to formulate a math problem, objects "but the train actually departs from our station at 10am!". Way to miss the point!
You don't need to dereference anything here as the empty string has length 0.
- Bangla according to the function has only few irregularities, rest is algorithmic transformation of the word given
- Many languages have a huge number of irregularities
- Many languages express case information somewhere else in the sentence-construct, so it's not just a word-transform.
This very much seems to try to replicate the machine-translation efforts of foregone ages.
> Wikifunctions will allow easy access to large knowledge bases such as Wikidata, but also to binary input and output files
https://diff.wikimedia.org/2023/08/07/wikifunctions-is-start...
> Using the simple facts housed in Wikidata, you will be able to write functions that make calculations, provide a person’s age, estimate population densities, and more, and integrate the results into Wikipedia.
https://www.wikifunctions.org/wiki/Wikifunctions:FAQ
> In the future:
> - It will be possible to call Wikifunctions functions from other Wikimedia projects, and integrate their results in the output of the page.
> - It will be possible to use data from Wikidata in functions.
This is a large part of why Abstract Wikipedia's goal is NLG, and almost all of the initial Wikifunctions facilitate NLG.
what is NLG?..
A lot of Wikipedia sites have scripts embedded in the wikitext which automatically generate or transform information on a page, e.g. automatically performing unit conversions to generate text like "I would walk 500 miles (804.67 km)", performing date math to automatically generate and update a person's age based on their birthdate, or querying structured data from Wikidata [1] to display in an infobox. One example of these scripts is the {{convert}} [2] template on the English Wikipedia.
Initially, these scripts were written in MediaWiki template logic [3], and were maintained individually on each wiki. This quickly proved unmaintainable, and some of those scripts were rewritten in Lua using the Scribunto extension [4], but these were still per-wiki, and there were frequently issues where different wikis would copy scripts from each other and introduce their own incompatible features.
The WikiFunctions project is an attempt to centralize development of these scripts, much like how Wikimedia Commons [5] centralizes hosting of freely licensed images and other media.
[2]: https://en.wikipedia.org/wiki/Template:Convert
[3]: https://www.mediawiki.org/wiki/Help:Extension:ParserFunction...
https://en.wikipedia.org/w/index.php?title=Template:Convert&...
This isn't even all of the code for the template, either; there were a bunch of subtemplates for individual operations. And now you probably have a decent idea of why Wikimedia wants to move away from that. :)
Templates like {{Convert}} are also by and large unrelated to Wikifunctions, which are initially and primarily more concerned about lower-language natural language generation problems, like cross-language conjugation, that are relevant to Abstract Wikipedia.
Or, in the Google.org evaluation I mention in a sibling: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or...
> Instead of attempting to invent so much from scratch, Wikifunctions could have been based on Scribunto. We think this is a huge missed opportunity. One of the longest-standing requests from the Wikimedia community has been to provide a means of reusing code across different projects by centralizing common code on a global wiki. This is a tractable, well-scoped problem (at least in comparison with Wikifunctions as currently conceived).
> “V1” of Wikifunctions could have been a new, central wiki for common Lua code. Structuring the development plan for Wikifunctions in this way would have meant a much shorter, surer path to delivering value to Wikimedians. It would have satisfied a long-standing community request, and (we suspect) earned the team quite a lot of goodwill. The team could then innovate on top of this initial version by building new graphical interfaces for producing and interacting with code, and by extending the capabilities of querying Wikidata.
> We are not sure why this was overlooked, particularly in light of the project requirements described above. In conversations with the Abstract Wikipedia team, team members suggested to us that this option was considered and rejected because Scribunto is an expert system and that Lua is based on English, and that these properties made it incompatible with the Wikifunction goal of broad accessibility. We think this is a bad argument, for the following reasons:
> - Lua has been successfully adopted by many projects whose primary language (as in, the language spoken by the users and developers) is not English. It is used successfully in hundreds of different language editions of Wikipedia.
> - Wikifunctions can provide localized, graphical user interfaces that generate Lua code behind the scenes (see Blockly, for example). Lua can also be used as a “transpilation” target for text-based programming languages (there is precedent for this on Wikimedia wikis). As an intermediate representation, Lua would be far more efficient and robust than an ad-hoc function composition language.
Which are all great points explicitly disregarded by Abstract Wikipedia for Wikifunctions.
I can guess the actual reason for this and it amounts to "managerial pride causing NIH". It's no big secret that the WMF is so overly ambitious with the projects it works on that it would throw away any and all prior art they've already made, just to make something that seems new (not to mention that many WMF projects kinda just quietly strand and are left to rot because of that).
I really wouldn't be suprised if some manager at the WMF decided that "no, using Scribunto for this is off the table", just because Scribunto is an existing project and they wanted the project to have additional complexity (so they wouldn't have to consider the viability of the project since a "demo" would then be years out).
The whole point of Wikifunctions is to support Abstract Wikipedia. Abstract Wikipedia deemed Scribunto/Lua as irrelevant ("out of scope", specifically) to their work.
The Google.org Fellows suggestion was to consider detaching Wikifunctions from Abstract Wikipedia, since Wikifunctions has value beyond Abstract Wikipedia's goals as a general-purpose community library of computing functions, but could still be used to serve the purposes Abstract wants from it.
Wikimedia said no to that suggestion, so here we are.
The project's response to the funding Google.org Fellows' evaluation is a deeper explanation that explicitly compares the goals of Wikifunctions to Scribunto (those Lua functions) — and is especially notable in the context of replacing templates and Lua modules since Google.org made a bunch of recommendations to further decouple Abstract Wikipedia's goals from Wikifunctions that Wikimedia thanked them for and subsequently ignored: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or...
The catalogue of functions useful to Wikimedia is much more prosaic than the {{Convert}} template and in line with the Google.org examination — it's more about Abstract Wikipedia's goals of natural language generation (NLG): https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue
The list of all functions is also similarly lower level; language transformations, form validation, sub/string operations: https://www.wikifunctions.org/wiki/Special:ListObjectsByType...
Much better description of what wikifunctions (the Wikimedia blog post seems more about where it fits in a broader strategy) is on wikifunctions about page:
Most programming languages make considerable effort in supporting old versions (gcc has c90 and c++98 for example, fortran goes even earlier). With Java, they make very strong efforts to support older code (as long as it uses the library which are labelled as stable). Old javascript code also tends to work well, as there are lots of old websites that don't want breaking.
Python on the other hand has gone out of it's way to nuke Python 2, and many new version of Python 3 deletes something from the standard library.
Also, I have Python 2 programs I'm keeping running, and expect to continue doing so for several years to come. Many libraries never made the jump to Python 3.
It seems like it should be a core functionality of such a project to easily set up complex computations as function compositions.
Wanna convert a string to the nato phonetic alphabet? Sure, that function is there... You gotta search for it first though. Does it accept lowercase inputs? No, not at the time of writing this comment... But you know, someone might fix that. No problem.
It sounds like a project that would be dead on arrival if it were launched by and for any other community. If wiki maintainers like this? Then great!
I do see value if people could integrate functions within wikipedia article. Maybe it makes sense for certain articles to include some dynamic widgets in them. Unsure I'd like that but it may work.
https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue
Does this have the potential to become some standard reference for code proving/transpilation purposes, or is it likely to end up like past attempts to unify all programming languages?
Speaking of which, are there already packages to access wikidata by objects' Q numbers? The two could work together quite well, with a bit of caching.
For the uninitiated - this is part of an attempt to basically turn WikiData[1] into article text. From what I understand, the idea behind this is to create a bunch of text rendering functions to make it easier to provide automatic translations of rendered text.
Don't get me wrong, the idea here is neat, but from what I understand the Wikipedia community has been having far greater needs not being met rather than some lofty goal to drive even more editors out of the site and replacing them with data entry drones. It's a bit too often that you hear about some ailing extension and the WMF response being "no budget for it", while a lot of it gets spend on projects like this.
[0]: https://en.wikipedia.org/wiki/Abstract_Wikipedia
[1]: https://www.wikidata.org/wiki/Wikidata:Main_Page - basically, Wikipedia data meant to be in a machine-readable format.
The most evergreen example of this is and probably always will be the rollout of VisualEditor, their WYSIWG editor. It was completely unwanted, was proven to be not ready for general use (due to all the problems that plague "combining a WYSIWG editor with a code view"), yet forcibly deployed on all Wikipedia languages without much room for warning or feedback. After that, the WMF promised communities the option to opt-out if they didn't want it... before violating it on the German Wikipedia because they voted to not roll it out. To my knowledge, VisualEditor hasn't improved over the years (beyond "rewrite Parsoid into PHP"), but editors have just gritted their teeth. That's the extreme example, but the WMFs relationship with building features for Wikipedia is... pretty much in a vacuum from what users want. A lot of their projects seem to be conceived by WMF managers first, without consultation to their existing community of volunteers.
Which is I guess fine for moonshot projects like Wikifunctions (since they don't involve the volunteers before release) but results in a lot of friction when a managers pet project is forcibly rolled out on all language Wikipedias. Very few of their big extension deployments have gone smoothly with the community. I distinctly remember the Minerva skin (mobile Wikipedia) needing tons of changes and improvements before it was usable as well, after it was already rolled out and caused a ton of negative fedback.
The Visual Editor works better now for its designed purpose than it did before, and rewriting Parsoid in PHP made it easier to deploy and maintain across projects (and in general Mediawiki). For editing text and non-templated content, which is the bulk of the work and the entry point for new contributors, it's matured into a useful tool.
The Visual Editor was also designed around English and European Wikipedia usage, and it still causes problems for other Wikipedia languages and for Wikimedia projects outside of Wikipedia. It has strong built-in opinions about the usage and design of skins and templates that other Wikimedia projects disagreed with.
And it's still frustrating as tech, both in bugs and by design, and especially when used in situations where wikitext editing is still common or required. It has caching and session bugs that either don't affect or aren't as destructive to other editing tools, or even pre-PHP Parsoid versions of VE. It still makes unnecessary wikitext changes — and arbitrary, if you weren't in the discussions where they were implemented or closed out as Won't Fix — to formatting and templates based on its own strong opinions. It breaks completely on interwiki redirects, a bigger problem for projects outside of Wikipedia than inside of it.
It's a useful, arguably invaluable, tool for new contributors that was badly designed at launch, has improved since, but is still fundamentally incompatible with many use cases into which it's been forced.
Perhaps, but so what. Wikidata was "a project with way too lofty ambitions" back when it started (a.k.a. make the Semantic Web something that actually works in a useful way) and it's been wildly successful. It now gets more edits per minute than the most popular Wikipedia and gets used all over the place by Big Tech firms.
The community is a means to that end, not the end.
Also if anyone hasn't already seen them, similar projects already exist and are more complete. E.g.
* https://programming-idioms.org/
Not to mention LeetCode, CodeWars, Project Euler, Exercism can kinda serve the same role.
> I love Rosetta Code and we took a look at it. IIUC Rosetta Code does not allow to run the code, or to compose existing functions together to create new ones. Also, Rosetta Code's aims to have complete sets of implementations for their tasks, which is not our focus.
Wikinews is fine. It's not original but it's also not harmful to have more in that space.
Wikibooks and Wikiversity seem like they should be merged. Same for Wikisource and Wikiquote.
Wikivoyage is just not well made and there are other better projects out there
Wikispecies should just be a result of Wikidata, not a new project. Catalogue of Life (taxonomy) is much more thorough, rigorous, complete, and maintainable. No academic would use Wikispecies over COL. Misguided hobbyists might and that's a problem. And Open Tree of Life (phylogeny) already (thoroughly) serves the purpose of centralizing phylogenetic studies so there's really no unique thing Wikispecies provides and it gets outcompeted on the things it does provide.
Wikifunctions... I'm gonna be honest. I really think it's not well thought out and wish it didn't exist. I know a lot of people have already poured in a lot of effort into it and I always hate to shit on that, but as a long time Wikipedian I hate to see Wikimedia's name backing this up. Especially when I feel there are already other efforts out there that are more complete, better thought out, and better organized that would benefit from more contributors
But yes I'm pretty sure that Wikispecies would not exist if it was proposed today, Wikidata has basically superseded it. The Wiktionary is a lot more popular but it could also end up being broadly superseded by Wikidata's lexical content.
Well, the first thing I expected to see when I clicked on a function was... the function.
It takes three or four clicks to get that, and the code is at the bottom of a fairly information dense page, where none of the information seems to have any relevance to using the function in your own code.
The other thing I expected to see is some discussion of the implementation, possibly allowing me to choose based on my needs (e.g. if I need to run a function 10,000 times each frame I might choose a different version than if I need to run it occasionally).
However, perhaps my expectations are wrong? Is this a tool for computer scientists to use in mapping the standard functions in various languages, rather than a reference resource for the working coder? The blog post doesn't go into any useful details, it's more like a standard marketing team announcement. They do mention it's part of wider initiative but it's not clear what that will be.
I do agree though that, especially in the age of AI autocomplete tools, I will never find this useful.
... actually, maybe it's a great resource for LLMs to learn from?
Neither of these. The video on https://www.wikifunctions.org/ gives a better explanation. If I understand it correctly, the project seems to be aimed at providing building blocks to more complex functions. Once the basic functions are defined in programming languages and described using natural language, you'll then be able to create new functions without writing any code, just by using existing blocks. Because each function is supposed to be described in multiple natural languages, you will be able to utilize them even if you don't speak English. Whatever you create you'll be able to embed on Wikipedia pages to perform calculations (e.g. automatic unit conversions) or string transformations.