The Web Data Commons project extracts all Microdata, JSON-LD, RDFa, and Microformats data from the Common Crawl web corpus
242 karma · joined April 15, 2017
The Web Data Commons project extracts all Microdata, JSON-LD, RDFa, and Microformats data from the Common Crawl web corpus
As far I understand, an improvement is to allows an URL like https://raw.githubusercontent.com/h-matsuo/uBlacklist-subscr... in settings.yml, so there is no need to update settings.yml.
Disclaimer: I am one of the maintainers of SearXNG.
It doesn't prevent people to hide / change the information, but at least by default it works even in a fork.
git rev-parse --abbrev-ref @{upstream}
when the server starts and then provides this link in the web pages?
The link will be updated automatically in the forks.
---
About the version, call "git show -s --format='%cs-%h'" to get the current commit.
How huge is the dataset?
It was imcompatibled with the Alcatel Speed touch USB (ADSL modem): after few minutes, the PC was rebooting. The driver was not SMP compatible. It took some weeks before I could change the modem, "hopefully" a friend in the same school year had the same motherboard and the same issue.
You can do that in this fork: https://github.com/searxng/searxng/blob/e839910f4c4b085463f1...
Note: since the version 1.0, searx stops sending request for 1 day when a CAPTCHA is detected which might help a little.
(I'm really interested by the results of your experiment)
Another science fiction example: the Replicators.
http://mappings.dbpedia.org/index.php/Main_Page
https://github.com/dbpedia/extraction-framework/tree/master/...
The HTML structure may change over time: if the request is executed few times over a long period, the scrapper may/will require more maintenance than the SPARQL request.
For example, the same wikipedia page 3 years ago is slightly different: https://en.wikipedia.org/w/index.php?title=Turing_Award&oldi...
Disclaimer: I follow https://www.youtube.com/channel/UCp2i8QpLDnWge8wZGKizVVw / https://www.twitch.tv/belett (mostly in French, sometimes in English).
Without these courses, I wouldn't have been able to write this request.
Some wikipedia infoxes are based on wikidata. I can't find an example, but here some links:
* https://commons.wikimedia.org/wiki/Commons:Wikidata_infobox_...
* https://commons.wikimedia.org/wiki/Template:Wikidata_Infobox
* https://en.wikipedia.org/wiki/Template:Infobox_person/Wikida...
There are lexeme too, and it is not based on wiktionary. Search the prefix "L:" (without quote).
Example:
* https://www.wikidata.org/w/index.php?search=L%3Acat&search=L...
* https://www.wikidata.org/wiki/Lexeme:L7
Also, there are a lot of tools on toolforge.org. One is reasonator which produce sentences from a wikidata item: https://reasonator.toolforge.org/?q=Q1339
* http://www.entitree.com/en/family_tree/Elizabeth_II
* https://family.toolforge.org/ancestors.php?q=Q187114
Tools found on this page: https://www.wikidata.org/wiki/Wikidata:Tools/Visualize_data/...
---
Some SPARQL queries: https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/...
---
Out of topic: I wish wikipedia would provide an API to get the infoboxes (made using Lua or wikidata).
(1) actually based on https://github.com/jivesearch/jivesearch/tree/master/bangs
* CKEditor: https://extensions.xwiki.org/xwiki/bin/view/Extension/CKEdit...
* Syntax: https://www.xwiki.org/xwiki/bin/view/Documentation/UserGuide...
Actually any wiki pages can define a Class and how to display this class and / or instances of a Class:
* https://www.xwiki.org/xwiki/bin/view/Documentation/DevGuide/...
* https://extensions.xwiki.org/xwiki/bin/view/Extension/App%20...
The Script Macro is useful to make some dashboards ( https://extensions.xwiki.org/xwiki/bin/view/Extension/Script... )
I've deployed this for the internal documentation inside a company I worked for (MediaWiki was a no-go even with a visual editor).
For each new feature, I was developing inside a clean new wiki, than I exported the changes once I was sure everything was okay. It is way more easy then to upgrade to the new XWiki version.
Quote:
> This puts browsers in the direct path of legislation in any given country’s demands. Features in browsers should be based only on user needs. Only when a law absolutely targets browsers should they do anything regarding legislation.
[Edit] remove the code block
I've been working for one of these CORDIS European projects : https://cordis.europa.eu/projects/en
The project was politically driven while most of the slides were selling dreams. As usual in this kind of project
- there was a schedule carved in stone
- the Conway's law gets a beautiful practical example
so the research becomes quickly a joke.
Some people:
- worked on out of topic subjects. At least no harm.
- criticized because of external/past/personal/out of topic reasons.
- criticized the work of others while doing literally nothing. My boss knew it, but since they are well known to wrote nice documents to get the fund for many years.
- worked very hard while being criticized through hidden communication.
- were here because of the data.
- were here to get the technology if by luck it ends up to be working project.
The usual burdens, I guess, but it basically kills the motivation (it is my experience in one specific project). In the end, people went to the different meetings to see Europe.
It seems some other projects are driven in the same way according to what I have heard.
It is only a sum up of experience. YMMV.
I sincerely hope that Gaia-X will be managed in better way.
- Alibaba’s Jack Ma Defends China’s 996 Schedule (12 April 2019): https://news.ycombinator.com/item?id=19645729 (scmp.com)
- Jack Ma defends the 'blessing' of a 12-hour working day(15 April 2019): https://news.ycombinator.com/item?id=19668671 (bbc.co.uk)
- Alibaba founder Jack Ma says staff should work 12 hours a day, six days a week(16 April 2019): https://news.ycombinator.com/item?id=19691793 (abc.net.au)
- China’s Grueling Formula for Success: 9-9-6(22 Feb. 2017): https://news.ycombinator.com/item?id=16573978 (wsj.com)
Example :
Feature-Policy: accelerometer 'none'; autoplay 'none'; camera 'none'; fullscreen 'none'
Documentation: https://developer.mozilla.org/en-US/docs/Web/HTTP/Feature_Po...The code is here: https://github.com/caddyserver/caddy/blob/master/caddyhttp/h...
The paper: https://jhalderm.com/pub/papers/interception-ndss17.pdf
> Since it lacks relocation information, it is loaded by the operating system at a pre-set address, at offset 0100h immediately following the PSP, where it is executed (hence the limitation of the executable's size): the entry point is fixed at 0100h.