Hacker News Reader

Comment by VikingCoder | Hacker News Reader

My mother worked grading standardized tests. It was a hellish job for many reasons (limited breaks, etc.)

One question she had to grade was essentially, "What's something you want your teacher to know about you?"

It was an essay answer, and she was supposed to grade it on grammar, etc. Just the mechanical aspects of writing. (The real question explained the details more, but that was the core of the question.)

She saw answers that would make you weep.

"My daddy touches me."

"I haven't eaten today. I don't know when I'm going to eat again."

Stuff like that.

And my mother was going to be the only human who ever saw their responses. Their teacher had no chance to see their responses, just my mom.

So she goes to her supervisor and asks, "What can we do to help these kids?"

The supervisor said there was nothing you can do. Just grade the answers.

View on HN

The US has federal child abuse mandatory reporting requirement laws which include teachers and school staff and personnel, as well as additional state requirements which vary but include, for 11 states, faculty, staff, and volunteers at public or private higher education institutions. Computer and IT professionals are also covered in cases.

Faculty, administrators, athletics staff, or other employees and volunteers at institutions of higher learning, including public and private colleges and universities and vocational and technical schools (11 States).

https://www.childwelfare.gov/topics/systemwide/laws-policies...

https://www.childwelfare.gov/pubPDFs/manda.pdf

This includes penalties for failure to report in multiple states:

https://www.childwelfare.gov/topics/systemwide/laws-policies...

so ... nobody wonders about the obvious rammification: then any ML scoring systems ... must detect child abuse signals!

Hey uh, that actually seems valuable.

I'd believe that ML could spot abuse that humans miss pretty well from signals like non-overt references in homework and school records, if one could come up with an adequate training set.

Much more likely than teaching ML to score reasoned and creative activity in any reasonable way.

What do you think the false positive rate is likely to be?

Depends upon what false negative rate you're willing to tolerate. ;) And I don't know how good of a signal there is. This is pure handwaving.

But this type of thing seems like the exact kind of spooky correlation that ML is good at spotting compared to humans.

Machine learning techniques are going to be absolutely awful in detecting something like this, the reason being it's exceedingly rare (at least I'm guessing it is; if we're talking about child sexual abuse by one's own parents, it sure sounds extremely unlikely- but even child abuse in general is probably rare [1]). Machine learning systems are awful at identifying rare events. Like the OP seems to suggest, the false positive rate would most likely be very high.

"Spooky" machine learning results happen when a correlation is abundant in a dataset [2]. Otherwise, machine learning techniques will probably miss it altogether.

______________

[1] Quick online search: https://www.inquirer.com/philly/blogs/healthy_kids/What-is-t...

[2] The archetypal spooky machine learning story is surely the one about Target sending baby item coupos to a girl in high school before her father knowing she was pregnant:

https://www.forbes.com/sites/kashmirhill/2012/02/16/how-targ...

Humans are awful at rare events and vigilance tasks, too. That's part of why we're seeing machine vision and machine learning starting to outperform humans in e.g. grading radiology screening scans.

The total incidence of child abuse of all types from infancy to adulthood is on the order of 1 in 3. This is not terrifically rare-- it's of higher prevalence than pregnancy and of positive screening events.

A much bigger concern is non-causative correlations. It'd be pretty easy to train ML to be racist or look for e.g. indicators of class, which are correlates of abuse.

As to false positive rates-- you can pick your false positive rate to be whatever you want it to be, by twiddling the threshold for a positive result. I'm not sure false positives are of that great of a concern, if the output from a system is a notification to school administrators that they may want to keep an eye out for this student.

> if we're talking about child sexual abuse by one's own parents, it sure sounds extremely unlikely-

Child sexual abuse isn't extremely rare and familial abuse is a very large minority of child sex abuse.

> But this type of thing seems like the exact kind of spooky correlation that ML is good at spotting compared to humans.

How? Particularly, where do you get training data at the required scale?

You take samples of hundreds or thousands of past students' schoolwork, e.g. submissions of essays for standardized tests.

You survey those kids in adulthood about whether and how they were subject to abuse and other types of relevant adversity.

You attempt to control the data so that you don't just latch onto other correlates of abuse (e.g. social class).

Who cares? The positives must be evaluated by a human anyway.

Who cares?

The people whose lives are ruined by being mis-identified by the system.

The positives must be evaluated by a human anyway.

Those same people whose lack of competence people are bemoaning throughout these comments.

> The people whose lives are ruined by being mis-identified by the system.

When a child writes "daddy touches me between the legs" in an essay, it doesn't matter if a human spots it or an AI that forwards it to a human, this needs to be investigated either way.

> Those same people whose lack of competence people are bemoaning throughout these comments.

It's not a lack of competence that's bemoaned, it's a massive amount of understaffing (and resulting overwork) in teachers and other school resources, as well as a drastic lack of financing because it's easy to cut budgets for schools for politicians as the effects only show up two decades afterwards.

Sprinkling some AI over it won't fix those issues, I'd argue it will make them worse as people blindly accept the results.

There were some cases in the UK about a decade ago where bugs in software the Royal Mail was using led to incorrect accusations of fraud. People actually went to jail over this, it took years to resolve.

> When a child writes "daddy touches me between the legs" in an essay, it doesn't matter if a human spots it or an AI that forwards it to a human, this needs to be investigated either way.

When a child writes a set of things that individually are not very concerning, they may have cues that could say "hey, this kid, you should maybe keep an eye out for evidence of abuse."

Particularly attuned, experienced individuals might spot these cumulative cues, but we all know that this is not all people dealing with children.

It's an interesting problem.

Humans will still let false positives through. And false accusations of child Abuse have significant ramifications.

This is so dangerous.

Society's bigotry is going to flood that bad boy so quick you might as well name it Gobbels.

I love ML. I want children to be safe. This is not the place for ML or AI or Quantum or any tech.

What needs to exist is better resources for those children, that mother grading the tests, the teachers of those children, and social services that are meant to support them. If you want to make a difference about this, look there.

Don't go building a automaton King Solomon who decides why this kid should be taken from these parents because speaking Spanish was worth -0.1 on some goddamn weight trained on data generated from a racist society.

This isn't a "spooky" correlation a cool algorithm can detect, it's a serious, layered social problem.

> Don't go building a automaton King Solomon who decides why this kid should be taken from these parents because speaking Spanish was worth -0.1 on some goddamn weight trained on data generated from a racist society.

Totally what I advocated for and not a strawman attack /s. Indeed, the chance that such an algorithm could be racist or classist and there being needs to avoid bad correlations and have appropriate controls is important.

I think there are opportunities here. Ideally ed-tech doesn't take humans out of the loop, but asks schoolteachers and administrators questions like, "Hey, are you sure students A, B, and C are being supported correctly for subject Z? Are you sure students D, E doesn't have some kind of abuse or other significant home problem? It sure looks like student F is in this subpopulation that research shows benefits from educational intervention Y. You might want to keep your eye out for that."

And then the teacher goes "Oh, crap. Now that I think about D, there were always these little things 1, 2, and 3 that seemed off... maybe this is worth a referral to social services to check on what's up."

Or "Oh, ... maybe F's struggles in reading really are a speech problem and we should handle that"

That is not how the law work. The law states that if people at a school is made aware or suspect abuse then they must act on that knowledge. A ML scoring system is obviously unable to be made aware or having suspicions, but the administrators could be help responsible if they happen to see something and chooses to not act.

It would be interesting to know if a child psychiatrist could be held liable if incompetence prevented them from seeing obvious sign of abuse, but I doubt that is covered under the cited law above.

Some of these will be 100% true as well. But don't make the mistake that there are no kids who go for shock value or are wantonly manipulative when they know it can't come back to them.

So how many are true and how many false? I have no clue. Literally none. And no it doesn't make me feel any better about the screams of existential agony even if that were a low percentage. Could be high too.

> But don't make the mistake that there are no kids who go for shock value or are wantonly manipulative when they know it can't come back to them.

In the US, school funding is based upon standardized test results, and bad results can shut a poorly performing school down.

It's drilled into every kid's head that these tests are very important, super strict and if they accidentally mess up, it can ruin their academics, because retesting and regrading are expensive.

As a kid I would go out of my way to fail those tests. The whole curriculum was designed around them, meaning that even if we did score high, any funding gains would just be put towards training us to take the test.

I thought the state was holding the school hostage, threatening to cut funds or shut them down if they ever stopped. We never learned anything about civics or American history. Until I was out of highschool, content regarding atrocities like slavery and the trail of tears was not on the test and that was enough to whitewash the whole curriculum.

Standardized testing is to the U.S. what lead waterpipes were to the roman empire.

Your school didn't teach slavery? What state?

It’s possible that there’s a difference between admitting it happened and an honest portrayal of the institution.

I don’t know about calling it manipulative. I remember taking the ACT, and struggling to plan out one of my essays. It was something like “tell us about a book that inspired you”. So I changed details about the plot so it all fit nicely and was easy to write. I can see something similar here, where someone takes on a persona when writing in order to effectively communicate.

That's entirely fair. Manipulation is kind of what a writer does yet the word manipulative has perjoritive connotations. Many types of writing don't have literal truth as any kind of pre-requisite. Others make a pretence of literal truth to achieve greater effect then basically lie, many autobiographies fall into this trap to some degree. All these things. Differential empathy. Data quality matters.

This is absolutely the case. In fact, my SAT prep class taught us that the factual veracity of our essays is irrelevant. Essay scores are almost entirely correlated with essay length as long as spelling, grammar, and basic paragraph structure (intro, body paragraphs, conclusion) is followed.

Of course some kids are manipulative, going for shock value, continuing an "in-joke", or just plain trolling. But would a teacher just look the other way, or would they talk to the kid? What would you want for your kids? This is why teachers assigning homework like "what do you want your teacher to know about you" and then not even seeing it is dehumanizing.

For the not eating, it's pretty easy to get data. It's like 1 in 5 children live in food-insecure households in the US and maybe 1 in 20 of those very insecure, so not eating before school provided lunch is common enough that if you're grading tons of papers you'll run into kids like that.

It could also be a student suffering from anorexia nervosa, which the confessional aspects of the essay would fit well with.

I'm confident that your example would of a less percentage than those mentioned in dmoy's comment.

https://www.ers.usda.gov/topics/food-nutrition-assistance/fo...

Food Security Status of U.S. Households with Children in 2017 Among U.S. households with children under age 18:

84.3 percent were food secure in 2017. In 8.0 percent of households with children, only adults were food insecure. Both children and adults were food insecure in 7.7 percent of households with children (2.9 million households). Although children are usually protected from substantial reductions in food intake even in households with very low food security, nevertheless, in about 0.7 percent of households with children (250,000 households), one or more child also experienced reduced food intake and disrupted eating patterns at some time during the year.

When I was a high school student, we had some state administered test in health class that tasked us with analyzing advertisements for liquor and tobacco and seeing if we could recognize harmful behavior that the ads might be promoting. This test had no impact on our class grade...

..which means students wrote whatever the hell we wanted. I was assigned a Captain Morgan (rum) ad. I wrote that the ad was glorifying maritime piracy and was likely responsible for pirate activity in Somalia.

False accusations can actually be the result of prior abuse. They may substitute one person for another. Or do things as a result of mental illness caused by abuse. Kids think differently to adults and may behave inexplicably. And unfortunately that means that an abused child is a terrible witness.

I was abused as a child (not sexually however) and I can attest to this. Many of my memories are highly charged and don't really hold up - they're very confused. Some of the scariest stuff that happened to me I don't even remember, and my siblings have had to let me in on it (and they were even younger at the time).

As a child you're really not prepared for the concept that your parents are treating you badly. So that realization doesn't come until much later.

‘Poor sentence structure and grammar, 1 point out of five. Sorry your daddy touches you.’

Punch up, not down.

boy do I hate that saying. It literally adds nothing to the conversation!

It can be a useful heuristic.

I don't find it adds much here.

You really misread that if you thought I was punching down. I was pointing out the absurdity of having to even grade such a thing.

Or...report it to the police? I’d gladly risk my job to do the right thing in that instance.

They should be mandatory reporters at least in the USA.

School contractors are mandatory reporters, but I suspect that may not qualify.

These would not be school contractors.

Why not? The school contracts with the testing agency that does the grading. Seems like a contractor relationship?

School contractor is someone who is hired by the school district on via a contractual relationship. Think temporary teachers, or custodian staff. It’s not a transitive relationship to every employee of every company who has some sort of contract, however small, with a school.

every relationship however transitive or small, is a relationship too! (Dr. Seuss)

So you're arguing that only individuals can be a contractor? That wouldn't make much sense, not only because such relationships are rare in schools. Most common are contractors that have been outsourced something like food service. The law would make no sense if it included practically no one. It would mean a company that provides, say, temp staffing within the school, and those temp staffers saw abuse, they too wouldn't be required to report. I have a hard time believing a court would rule the definition to be so narrow. Both the common language understanding of the term and legal literalism would point against that. There's no transitive property here. We're not talking about contractors hired by contractors hired by contractors. We're talking about a contractor and its employees. There is no way for it to exercise this reporting requirement save through it's individual employees.

Depending on the state, yes, they are, for 11 states.

https://www.childwelfare.gov/topics/systemwide/laws-policies...

Hand-graded standardized tests are usually anonymized.

Someone could track back the number to the testee, they need to get their grade at least.

It takes a five minute phone call with the company's legal department or a warrant to find out who the kid was. Either way it would need to be escalated to involving law enforcement.

What my mom saw had an ID number on it. No other demographics. And she was grading from multiple states.

So do what? Contact her local police?

With a written accusation from a child? Is that enough to get a warrant to force the company to release the demographic information?

And people don't work at a job like that because they want to. They work there because they need the money.

Everything she took in and out of there was monitored, too. So it's not like she can go to the Xerox, and walk out of there with a copy.

It's beyond dehumanizing. For everyone. The kid, the people who work there.

Yes, contact the local police so an investigation can get started. Yes, that’s enough to at least report it.

> So do what? Contact her local police?

> With a written accusation from a child? Is that enough to get a warrant to force the company to release the demographic information?

Absolutely! My girlfriend works as a counselor at a school and she is required by law to report all serious abuses by parents.

> So do what? Contact her local police?

Collect or photograph all the evidence, record every conversation with supervisors, escalate as much as possible internally, then contact local police, and at the same time go to the media. Don't quit, but if necessary let them fire you and then sue. None of this is easy.

When escalating I'm sure it'll be effective to say it'll be an interesting story for the news and how the incident is being blocked by supervisors that encourage child abuse.

If she did Xerox the disclosures, walked out, and said "please call the cops" when challenged, at least it would be a matter of public record

> So do what? Contact her local police?

> With a written accusation from a child? Is that enough to get a warrant to force the company to release the demographic information?

Yes, she should have absolutely went to the local police. A child's first hand account in writing of child abuse and neglect is slam dunk evidence to secure a warrant to link the essay ID to the individual child.

> Everything she took in and out of there was monitored, too. So it's not like she can go to the Xerox, and walk out of there with a copy.

Doesn't matter. She could have went to the police herself as a witness. That alone would be enough for a probable cause warrant to retrieve the essays.

It is very sad she saw these signs of abuse and did not report it.

Tell you mom to take those numbers and the company's name to the police. They can walk back the identification problem.

Reply on news.ycombinator.com