19 karma · joined January 8, 2015
You may want to try ParslePy, it combines CSS/XPath functionality, allowing you to declaratively specify the selector paths in a JSON file. I just made a PR to allow YAML over JSON, but not sure if Pip picked up on it yet.
They then demonstrate a sample query on this dataset, called 'connection mining', identifying people who have enough data in common with a given person.
Now, although the Datanami article does not particularly state the Beijing government as making use of TigerGraph's services, it does mention his clients as including 2 Chinese state-owned companies, State Grid and China Mobile.
Given it's been established his clientele includes Chinese state-owned companies and no-one but the Chinese government has the data described in this 'Social Graph' data set though, let's say this may have been made for Beijing. Sounds like they might be using this query to track dissidents by association.
I tried finding what TS can do now, and figured out type-level tuple iteration, among a few others. The current roadblock seems to be getting function return types. Progress, for anyone interested: https://github.com/Microsoft/TypeScript/issues/16392
At this point I'm amazed how close we are to typing anything, despite having only 5 (!) type-level operators, with their respective warts.
I imagine in a visual environment, being able to make useful suggestions on potential ways to use/combine different nodes/types would help as well as an auto-complete for the user's intent.
I'm actually also a bit reminded here of MS Excel Power Query, which also offered a GUI for data transformation, see e.g. [this pic](https://blogs.office.com/wp-content/uploads/2015/07/6-update...).
I bring this up because I see you covered visualizing the steps, while they focused on showing the data (though after finishing a transformation the script could be generalized into a reusable function). I wonder if adding a dimension like that could be helpful for Luna as well.
If you target non-programmers, showing things as concrete as possible (e.g. their data transformed by whatever function they just pulled together) sounds like it might help make things even more accessible.
I saw it also supports both row-oriented and columnar modes. Does that mean this is meant both as a transactional and as an analytics-oriented database?
My understanding is that serialization became a thing because in-memory representations tend to use pointers to shared data structures that may thus be referenced multiple times while being stored only once. This would not translate 1:1 to serialized representations (where memory offsets would no longer hold meaning) -- much less in any language-agnostic way.
So I have this suspicion that Apache Arrow would not support reusing duplicate data while storing it only once. Would anyone mind clarifying on this point?
By writing to the model, you mean programmatically adding new measures or the like?
My interest is in programmatically querying models using DAX, though to this end I'd also look to look in the direction of Microsoft's DirectQuery mode in SQL Server which supposedly did DAX-to-SQL conversion.
If one could use such a conversion plus MDX to start querying models on an Apache Spark cluster through pivot table/chart interfaces...
I mean, point taken, but in case you were wondering, then yeah. VBA also does SQL operations on Excel tables, but that's, well, worse.
To expand on my question, I'd heard (from Rob 'PowerPivotPro' Collie, former product manager on the project IIRC) that the core had been written in 'unmanaged code' (probably C++?), so I believe reverse engineering it would be a significantly larger effort than just opening some of its DLLs in say DotPeek, at least from as far as I've been able to tell.
Interestingly, the user-written code (around 175 lines) appears to make for less lines than the Angular one. It seems that doesn't happen much, so to me that plus this using React makes it seem notable.
Then again, Tuxx using FB's React + Flux without otherwise having affiliation with FB may raise questions about how well FB (React/Flux) will play with them in the future...