Here's how an IA tool may flag parents with disabilities

Post Reply New Topic

Here's how an IA tool may flag parents with disabilities

Social services already discriminate against people with disabilities. So, the AI will pick up on that pattern in the training data and so will reproduce the same biases.

Will the AI also flag junkies, drunks, and abusers?

The entire article is worth reading. I’m not sure AI is the right term here. They are just using an algorithm.

Of particular note: The algorithm used to determine when to remove a child from parents is secret. Even attorneys for the parents cannot see it.

And: algorithm can include things like race, juvenile records, growing up in poverty.

Finally, this is spreading world wide - Australia, Denmark, …

We don't know what kind of imperfect rules this type of tool is replacing. We are only hearing one side of the story about this couple. I would bet everyone in government would love this IA tool idea since it takes the imperfect decisions they inevitably make largely out of anyone's hands. The problem with giving anyone on the parents side insight into the algorithms, is that the worst parents are going to be trying to game the system and would use any such information to do that. If enough information was available to recreate the tool, I could imagine some lawyer doing that and providing guidance to parents who claim they were unfairly flagged.

It's an interesting dilemma in the child-protection vs parents-rights debate. If independent contractors are writing the code for these programs, certainly government oversight is needed to insure quality. This article does not address that issue.

Another issue is parents being untruthful and often getting the kids to lie too. People often think they are much better at assessing truthfulness than the actually are.