spothedog1

New S3 Table Buckets [1] do automatic compaction

[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tab...

Definitely do not start with Jena/Fuseki, pain in the ass to set up. Start with Oxigraph or rdflib in memory to play around with how to query/interact with the graphs

yup I've been working on building a platform entirely built on RDF Ontologies. Excited to try to combine it with LLMs in the not-to-distant future.

Say I have a news article that describes a Government agency with a few departments and people who work in that department. Graphiti gives 2 options

1. Use whatever node and fact schemas Graphiti comes up with will be different everytime because it's using a non-deterministic LLM

2. Input my own schemas

1 will not be standardized deterministically, 2 will require a lot of work from me to figure out how to structure an organization. There are a million edge cases to think through

I think a big value add would be to just automatically use the Organization Ontology [1] and everyone who uses Graphiti on a text block that describes an organization will get that back.

Then defacto everyone's schemas can interact with each other. My business value prop is not going to be in defining an organization's structure, but it's still useful data for me have. Forming good Ontologies takes a lot of work.

I was thinking of just fine tuning my own LLM with RDF ontologies so it always returns graphs formatted in commonly used RDF schemas

[1] https://www.w3.org/TR/vocab-org/

EDIT: Just saw your edit 2 comments above. And yes exactly that is what I think would be useful. I'm looking for a project in this space that is opinionated cause frankly using a graph with a random assortment of node and edge types everytime I run it is not very useful.

Can you explain why you think Dublin Core is a bad standard?

Very cool, I've been working on getting Qlever setup and using it as my main triple store. Was excited to see this hit my feed

The reason I bring RDF is because I use ontologies that have been defined by experts and covers ton of edge cases. If a group of genealogists define a `fam:` RDF Ontology and publish it, then I want every family relationship in my graph to use their Ontology.

I'm looking for something like graphiti that can take in a text block and when creating the relationships, automatically know to use the `fam:` ontology when creating familial relationships. The vast majority of people don't feel like defining schemas for every little thing and they're basically the same across all systems except for custom proprietary ones you define as your IP.

Their ontology would have OWL rules like `fam:isChildOf` `owl:inverseOf` `fam:isParentOf` so running an OWL Reasoner over the graph would generate the inverse triples as well

So if I had the text `Joe is Bob's dad`, input it into graphiti, then get the triples

person:Joe fam:isParentOf person:Bob person:Bob fam:isChildOf person:Joe

and the edge would be in a shared definition amongst all graphiti users. The LLM can be fine tuned to recognize exactly what fam:isParentOf means so there is no ambiguity. Right now I'm guessing graphiti could spit out edges `IS_SIBLING_OF` `SIBLING` `SISTER` `BROTHER` etc, its not standardized which makes it difficult to interact with computationally if say, I wanted to input a bunch of random text and then run pre-trained graph models of family networks.

Looks cool, would love support for RDF Graphs. The reason I prefer those is because the ontology is already well defined in a lot of cases which is 80% of the battle with Knowledge Graphs in my experience. Without a well defined Ontology I think LLM <> KG integration will not live up to its potential. LLMs have to know what nodes and edges really mean across diverse datasets

Following up: A reason I really like using Jupyter in PyCharm is because Github CoPilot works in it which helps a lot.

I'm very interested in this. I'm a Software Engineer who's been doing some Data Science on the side and been looking for something like this.

My current set up is running Jupyter on an EC2 instance and using inside PyCharm. One feature I actually really value is being able to use it directly in PyCharm as I can have my IDE on one side of split screen and my browser on the other. Not sure how feasible it is to integrate something like this into an IDE, VSCode would work

But a real killer feature that could get me to switch to a browser based would be the ability to load custom context about the data I'm working with. So I have all my datasets and descriptions of all their columns in my own database and would love a way to load that into the LLM so that it has a greater understanding of the data I'm working with in the notebook.

I store all my data in objects called `distributions` [1] and have a `get_context()` function that will return a text blob of things like dataset description, column description, types, etc.

The issue with all these auto-code AI tools is they don't really have a good grasp of the actual data domain and I want to inject my pre-made context into an LLM thats also integrated in my notebook.

[1] https://www.w3.org/TR/vocab-dcat-3/#version-history

twitter only shows the first tweet if you're not logged in, can you post the repo here

Can you expand on Postgres vs Graph Databases?

This is what I've been thinking for the past year. Do you have any more resources where I can explore what you're talking about?

* What are some deductive reasoning engines out there?

* How are you storing the rules in a graph? Using triples?

* What query languages are you using for this?

My personal exploration has me moving in the direction of using LLMs as a Language Model and some sort of structured knowledge representation as a Knowledge Model. So using knowledge graphs to store facts and data and LLMs to interface with it. I wish there was more focus on this, right now I'm just using RDF graphs and SPARQL to try to create my own knowledge engine.

I live in DC, what meetups do you run? I would love to join

what projects are trying to use knowledge graphs to retrieve truth? i've been playing around with that approach. how do you encode your own "truth" that may be different from anothers?

Ebikes are pretty powerful, kinda like riding a smaller motorcycle

You could design cities so people could walk to local neighborhood shops instead of ripping coal on a highway to get to Walmart. Skip public transit, just a local bodega in a lot of neighborhoods would cut car trips by a ton.

Living in a newly built high rise in more attractive to some people than a decades old house that might not be in the best shape. You can't "redirect demand", that's just not how the world works. When a bunch of jobs are centered in a city, then people will move to that city to work. You need to have a solution that can accommodate the population growth. Just telling people to go away is xenophobic and denies the reality of the situation.

Building new housing isn't about "giving rich people a cozy pad" its about allowing millions of people from geographically disadvantaged areas live in high opportunity areas. I grew up in a small farm town 3 hours from the nearest city so I'm just supposed to live there the rest of my life? I'm not allowed to move to a city cause I got a good job offer? Sorry but your attempts to dehumanize new arrivals isn't going to convince many people, what about the millions of immigrants who come to this country, are they just banned from living in areas with jobs under your scenario because some people already live there? You're still not providing a solution to this problem, you've just ignored it.

You also must have completely ignored my comment from above, if you don't build new housing those existing people are just going to get kicked out at an even faster rate if you don't build new housing. New housing acts as a sponge to soak up demand. The entire reason housing is expensive is because we don't build enough housing units in places people want to live. Your solution to not build anymore only makes it worse. Their rents will go up at an even faster pace. If you made building fast, cheap and easy then housing units at all price points would sprout up. Do you think if you don't build a high rise then wealthier people just go away? No they just look at the existing housing stock and buy up and renovate whatever is there. Houston has one of the most deregulated housing markets in the country and for this reason the average home hasn't gone up in price despite a fast growing population. You have to build to keep up with demand.

Renters and owners are different, in a gentrification scenario owners get very wealthy. Tons of poor families who own in poorer neighborhoods just received a massive cash infusion when a developer bought their parcels. Gentrification is one of the largest transfers of wealth from rich to poor in history. Your hypothetical of someone selling their house for a huge premium and then having nowhere to go doesn't bear out in reality, they've just become rich from selling. There are millions of stories of poor families buying cheap property on the outskirts of town decades ago only for that land to worth a fortune now due to development potential.

Renters on the other hand suffer the most from lack of building, you say when they get kicked out they'll have to pay more in rent. That's because housing isn't being built fast enough so prices are skyrocketing. When you have increasing demand and fixed supply, prices go up. The only way to keep rents stable is too build as much as possible. As for neighborhoods, places change. You can't keep them stuck in time for ever. Families move, economies grow, people age and we need to have solutions that adapt, not ones that pretend no changes are happening and try to keep everything in place.

So in short you still haven't provided a solution to people wanting to live in areas with jobs. These aren't rich people with vacation condos, these are people who come from areas without good jobs and want to live in areas with jobs. Please give me a solution that accommodates them or you're just plugging your ears. Trying to deny reality isn't a solution and will only make things worse.

If a developer is stupid to overbuild, then let them take that risk and that loss. We don't have to zone away risk. As for money laundering, you're using an example of one Trump Tower out of millions of apartments buildings to say it's "more likely" that every apartment building built is just used for money laundering. That's just silly.

As for structural safety, no one is arguing to reduce building codes or safety regulations so that's just a straw man argument. If an apartment building is built unsafe then the government should have done a better job inspecting and enforcing/expanding their regulations, it's not really an argument to not build.

High rises are built because people want to move to places with better opportunity. Should we simply not allow anyone to move to places with high opportunity? Are people who were born in rural areas or small towns not allowed to seek a better life in a big city because some people already live there? If you think people people should be allowed to move to places with better opportunity, then you need a solution on how to house a growing urban population, and that solution is building denser and taller high rises and housing of all types. You seem to view the people moving into the high rises as some sort of evil gentrifier trying to make other people lives hell, but they simple want to live in a more prosperous location. How is your argument any different than American anti-immigration people saying no one can immigrate to the US because "its full". Simply put, you need to build more housing in places where people want to live and unless you want endless urban sprawl of low density houses then you need to build high rises where demand supports it.

I'm not trying to argue in bad faith, but I genuinely don't see a solution where you allow people to move to areas of high opportunity without building more high rises (or just denser housing in general).

If the demand exists to build high rises then those people would already be pushed out at an ever faster rate. If you take a plot of land that previously had 4 units on it and turn it into a high rise with 100 units, then you can fit 96 higher income people onto that plot of land. In the scenario where you don't build that high rise, those 96 higher income people simply go into the existing housing stock and push those people out anyway. You can think of a high rise as a sponge soaking up demand. In our hypothetical with the high rise, the existing neighborhood's housing stock is in less demand (and cheaper) because all the wealthier people moved into the high rise.

Imagine 2 empty parcels of land in downtown Manhattan, one can have a giant skyscraper of any height built on it and the other has to be a single family home. The parcel of land with no restrictions is going to be worth exponentially more because you can built a massively profitable structure on it, the other one is so regulated that it isn't worth any where near as much. The more profit you could derive from a parcel of land the more its worth. Now I used an extreme example just to get my point across but the same principle holds on a small scale. If your house was rezoned for apartment buildings and the demand existed, someone would come in and buy it off your hands for a huge premium. Since in our hypothetical someone had just built an apartment building next to your house, the value of the land is already high, simply allow your parcel of land to allow more structures would cause it to rise in value.

On the infrastructure side, its a much better deal for you as well, the combined value of all the property taxes from the apartment buildings will be way more than if that lot had remained a house. Since the road, water, sewer and electric lines are already in place nothing new has to be built. You benefit from all those property taxes coming in from the apartment building while the liabilities to the government have barely increased. The local government can use this new surplus of taxes to build new amenities for you.

For traffic, sure but that's why you want to build more pedestrian friendly neighborhoods and public transit so people don't have to drive everywhere. Once you have enough people in a place you could open small shops so they can hang out around the building rather than drive everywhere.

If someone built a gigantic apartment complex next to your house it would mean that the demand for housing in your neighborhood was massive and restricted by zoning. Your land value would skyrocket by allowing apartment buildings and you could take a huge pay day and move somewhere farther out from the city. You shouldn't get to halt US GDP growth in its tracks because you want to dictate what your neighbors do with their property.

The quality of this comment section is like Facebook or Reddit level. Practically no deep discussion of the actual topic at hand just dunks on Zuck.

You can disagree but it's true, the melting pot isn't a myth. I'm a brown skinned child of immigrants and my entire friend group is white and black people who have been here for generations along with other children of immigrants. Even my anti-immigrant Trump supporting next door neighbors are like a second family to me. Again for the vast majority of the country people get along amazingly well regardless of background, you just hear about the bad parts on the news. You can keep thinking people don't integrate into society but after 1 generation of being in the US anyone from anywhere can safely call themselves American. I can't say the same for Greece but culturally in the US wave after wave of immigrants become fully integrated American quite fast.

The average energy usage per person in the USA has been going down in the past few decades as appliances and electronics get more efficient. It will continue to go down as more things electrify and shift to renewables. We can decrease resource consumption through smarter growth policies like building denser, public transit while growing our population for economic benefit.

You're wrong that low-skilled immigration is a net loss for the USA. Low skilled immigrants contribute more than they take out by a good amount. They increase demand and grow the economy. Additionally their kids grow up and out earn them drastically being huge net-positives.

Not sure where you got that idea but I definitely don't think all immigrants coming here are rich, I'm a huge advocate for more poorer/low skilled immigration. I've outlined in a bunch of other comments why exactly but for them if their life struggling in the US is better than where they left and they are voluntarily here to give themselves and their children a better life it's good with me and a huge benefit to the USA.