Full text search on 400M US court cases

the frightening part is although your father's record is available to the public, police officers who are caught lying while testifying get to seal the record.

in many jurisdictions, sealing a record is the equivalent of destroying it.

your crimes will haunt you forever because the system never forgets, meanwhile they simply go back to business like it never happened

ref https://www.google.com/amp/s/www.nytimes.com/2018/03/18/nyre...

Found an assault charge on my Mom from '92.

Arrest for a speeding ticket? Good God.

apparently the search box repopulates from a cookie. Last thing I search was my name. I clicked your link and freaked out for a second.

"In Nix v. Hedden, 149 U.S. 304, 37 L. Ed. 745, 13 S. Ct. 881, the question presented was whether tomatoes were to be classed as fruit or vegetables under the tariff act. The court found no particular help from the witnesses called, and decided the point through the use of a dictionary, which was not evidence, but an aid to memory and understanding."

https://www.judyrecords.com/record/5sfv13kuy1081

Compelled prostitution of a person under 17, duh. I mean, obviously. By a person named "bitch" in "whoresville, usa". It was all authorized by "PIMPDADDY", as shown there, plain as day.

I'm going to go out on a limb here and say this is probably a test record for whatever software they use.

Ahh yes, the John J. Bitch from Slutville who lives on Whore Street. I know him well. But I don't remember him having pink eyes or being 335 pounds.

That is amazing.

There are lots that are amusing, with most names or insults getting a hit. It’s seems the ‘AKA’ field has all sorts of entries.

The general idea has various problems. For example, would newspapers or accounts of things/people, in various media, of objectively public information be required to be retroactively removed from any mention? Does it make sense to force and dictate what entities/individuals can do with basic information at the discretion of anyone who doesn't like it? Just a few thoughts. The records exist in the database because they are public information. If a record is removed from public view, that's done when requested because it's the right thing to do, although there is no strict legal obligation to do so.

I found my conviction for assault from a bar fight 23 years ago. Since then I quit drinking, went to college, raised a child who is now 20, and turned everything around. It's pretty disheartening to see it can be found by anyone 23 years later. Unfortunately, I'm not surprised in the least that we allow this in the US.

Elasticsearch is a search platform. A "database" but meant for search stuff. It's not part of MySQL.

Elasticsearch, Lucene under the hood, implements an inverted index which is an extremely fast data structure for text search. ES has clustering as a primary feature too and many search features that can significantly improve relevance that you won't find in MySQL and most other databases.

The gist is Elasticsearch is a full-index database. Whatever data goes in gets indexed as compared to only indexing certain fields in MySQL on which you perform search frequently. Think of Elasticsearch as MongoDB + full-indexing. It's a document storage with blazing fast search and aggregation.

I've noted CourtListener on the info page: https://www.judyrecords.com/info

"PACER notwithstanding, CourtListener is the most powerful case law research tool available online — and in many ways is much more powerful."

This is based on CourtListener's 4 million+ written court opinions, which judyrecords has recently integrated. But you're right, CourtListener has more case law research features.

Interesting point. If you know the state where someone lives, you can look up the same info on the government website. Additionally, many many other databases have same public data but they ask for a payment to search.

Is the case available on the court portal? You can email me richardbarosky@gmail.com if its been removed since it was retrieved.

The race data in court case records is very often used for crime statistics. It's probably the most analyzed data point after what the incident was about.

No it’s likely not, arrest records are separate from traffic citations and are two different databases. Also, your race may come from the cop filling out the ticket, or it may come fr9m your license in more advanced jurisdictions. The source for crime statistics is usually not court records, those are held by the courts.

Many similar databases and people finder sites are behind a paywall. There are a lot of positives to being able to use public records data available to make more informed decisions, whether it's to let your kid stay at someone's house you don't know or whatever it might be. Thanks.

15KB is maybe the average case size, including HTTP request data.

That's 1024 * 15 * 439,000,000 = 6.7TB roughly.

The cases are all compressed, so I'm not using 6.7TB non-compressed for cases. But there are other request and non-request related records needed too. Just my backups currently.

I've downgraded from that. I talked about that in that post. It was most definitely a knee-jerk reation to getting slashdotted on a popular subreddit and not wanting that to happen again. However, still on some very good hardware and handling current workload pretty well right now. That estimate was high.

333 N Warcraft Lane Undercity, Washington 99999

Looks like a place I would like to live in.

Only 3 pages are indexed on Google. Actually, most of the other legal databases (listed on info page) have their cases indexed on Google. However, judyrecords cases aren't indexed on Google. I understand your general sentiment.

Worse, there's no obvious business model or disclosed funding source or institutional affiliation here.

That leaves me with the distinct impression that they're monetizing data about visitors and searches in some horrible way. (Data targeting for mugshot shakedown operations?)

I'm not going near this.

Sometimes even seemingly trivial cases can be caselaw precedent that people should be able to see and access without paying (they are public records).

If our society decides it is necessary to act with the full weight of the law behind it, then it would seem better to have the information available for the public to verify than not. I'm not saying it is all great, but that it is far better to have information available so that things like average sentence length for a given crime based on demographic and psychographic information can be queried by all. If a city that is 50/50 male/female and 20/80 black/non-black finds their speeding tickets are 70/30 male female and 35/65 black/non-black, then it may be worth investigating to see if police are being fair who they give warnings to, who gets reduces tickets, and who gets neither.

As for major privacy concerns, it is generally the more major crimes that have the larger issue with the victim being known. Knowing that some one was the victim of mischief vandalism is far less a privacy invasion than knowing they were the victim of sexual assault of a child (and even hiding the victim's identity often doesn't do more than hide the name from a passive search).

Then there are the benefits that other posters have raised, such as being useful for knowing past decisions used even in minor trials.

Many Zoom meetings are recurring and this might not be safe

wow...

"The Court finds that New Century had access to the SQL Data [pg. 536] Structures and that there is enough probative similarity to find that New Century factually copied the SQL Data Structures."

The next question might be to have 'Positive Software' demonstrate that they did not, in fact, take their table schemas from some place else. Like... textbooks? Or... example database schemas from vendors. Or tutorial sites? Or competing products?

There may be something extremely unique about part of their structure, perhaps, but... at the same time, there's often very little variety in how most similar data (crm/sales/lead gen/etc) might be stored to be remotely usable for reportin anyway.

"misappropriation of confidential information". Without seeing the structures in question it may be hard to say, but typically 'confidential info' is qualified with "not elsewhere available"-style clauses.

"... Likewise, the Court finds that there are more than one or a few ways to organize the data structures required for programs such as LoanTrack and LoanForce..."

Yeah, but usually there's only one good way to do stuff. Yes I could just have one row with 940 columns - technically, I could make my program work with that - but it's extremely suboptimal - regardless of whether I've seen anyone else's table structures or not.

It's 80 cases when searching: "donald j trump"~4

This is a proximity search, to ensure it's actually turning up one of the various permutations of the name (as different court protocols may refer by surname first), rather than documents that just happen to contain each of the terms somewhere.

For fairness, "hillary rodham clinton"~4 turns up 193 cases.

Relevant doc: https://www.judyrecords.com/info (down the page, under "proximity search")

I've mentioned other legal databases on the info page. It's public information. judyrecords is the largest free database of court cases, but there are many other free/not free ones as well.

Interesting. Haven't seen links being used in a while. Thanks for posting.

Updated and improved

    #!/bin/sh

    j=https://www.judyrecords.com;
    case $# in
    1)
    tmux new -P -d links;
    tmux set set-remain-on-exit on;
    tmux send g;
    tmux send $j/addSearchJob?search="$@";
    tmux send c-m;
    sleep 1.7;
    tmux send d;
    tmux capturep -p|sed -n /./p;
    tmux send g;
    tmux send $j/getSearchJobStatus;
    tmux send c-m;
    sleep 1.7;
    tmux send d;
    tmux capture -p|sed -n /./p;
    ;;0)
    test $n||n=1;while true;do test $n -le 200||break;
    tmux send Down 
    tmux send Down 
    tmux send g 
    tmux send $j/getSearchResults?page=$n 
    tmux send c-m;
    sleep 2;
    tmux send Down 
    tmux send Down 
    tmux send Escape;
    tmux send F 
    tmux send v 
    tmux send c-u 
    tmux send 1.htm 
    tmux send c-m 
    tmux send o 
    sed -n "/href=\"\/record/{s,.*record\/,$j/record/,;s,\",,;p;}" 1.htm;
    __grepq=$(exec sed -n '/a class=\"goToNextPage/!d;=;q' 1.htm);
    test ${#__grepq} -gt 0||break;
    n=$((n+1));
    done;
    tmux killw ;
    esac

Thank you

So the data should be public (as it always has been), but it should not be easily accessible?

Looking at the results, those all appear to be from CourtListener's bulk data.

There are only 532 cases, so it's not too bad.

From Reddit thread:

> MySQL 8 is used for DB. The seach server uses elasticsearch 7.8.

From other comment: CourtListener has about 4 million opinions, which are included. On top of that, 435 million additional cases from throughout the US.

Good point, that would definitely be an improvement. Thank you.

Thanks. All the data is collected from various government databases.

CourtListener has about 4 million opinions, which are included. On top of that, 435 million additional cases from throughout the US.

Thanks, as far as I know it's the largest database of court cases on the Internet. If there's enough traffic I'll support the site with ads. Don't have any other specific plans currently.

Yes, it has records from different trial courts.

It is all be public records. The source of the original data is the court system. If a 3rd party physically scrapped it from the court system, others should be able to digitally scrape it.

All the data is from government databases directly, aside from CourtListener, which was recently integrated. It would be good to specifically mention CourtListener's contribution.

I recently added advanced query support. Looks like I need to clean up some validation. Thanks.

Interesting, I'll check it out. Thanks for the link.

From what I understand, he had some kind of academic library access for PACER and used that to bypass what others would be changed for. There are lawsuits against PACER charging fees for what's public information generated by taxpayer money. He ended up being charged with various crimes related to maybe computer fraud and eventually committed suicide. A very sad story.

See other comment link to reddit.

I don't think you understand what you're talking about. There are many databases that are made up of public records. Many aren't free, some are.

> lol so many records that should have been destroyed and not indexable!

You want secret courts?

Thank you

Thanks. I saw your post on reddit a while back. Was going to ask about your tech stack.