This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
How NOT to Regulate the Tech Industry
Hot on the heels of my comment describing the UK's effort to finally rid the IoT market of extremely basic vulnerabilities like "has a default password", Colorado jumps in like Leroy Jenkins to show us how, exactly, tech regulation shouldn't be done. SB 205 is very concerned with "algorithmic discrimination", which it defines as, "any condition in which the use of an artificial intelligence system results in an unlawful differential treatment or impact that disfavors an individual or group of individuals on the basis of their actual or perceived age, color, disability, ethnicity, genetic information, limited proficiency in the English language, national origin, race, religion, reproductive health, sex, veteran status, or other classification protected under the laws of this state or federal law."
Right off the bat, it seems to be embracing the absolute morass of "differential treatment or impact", with the latter being most concerning, given how incomprehensible the similar "disparate impact" test is in the rest of the world. This law makes all use of algorithms in decision-making subject to this utterly incomprehensible test. There are rules for developers, telling them how they must properly document all the things to show that they've apparently done whatever magic must be done to ensure that there is no such discrimination. There are rules for deployers of those algorithms, too, because the job is never done when you need to root out any risk of impacting any group of people differently (nevermind that it's likely mathematically impossible to do so).
Their definitions for what types of algorithms this law will hit are so broad that they already know they captured far too much, so they go on a spree of exempting all sorts of already-existing things that they know about, including:
If your idea for a mundane utility-generating algorithm didn't make the cut two weeks ago, sucks to be you. Worse, they say that these things aren't even exempted if they "are a substantial factor in making a consequential decision". I guess they also exempt things that "perform a narrow procedural task". What does that mean? What counts; what doesn't? Nobody's gonna know until they've taken a bunch of people to court and gotten a slew of rulings, again, akin to the mess of other disparate impact law.
Don't despair, though (/s). So long as you make a bunch of reports that are extremely technologically ill-specified, they will pinky swear that they won't go after you. Forget that they can probably just say, "We don't like the look of this one TPS report in particular," and still take you to court, many of the requirements are basically, "Tell us that you made sure that you won't discriminate against any group that we're interested in protecting." The gestalt requirement can probably be summed up by, "Make sure that you find some way to impose quotas (at least, quotas for whichever handful of groups we feel like protecting) on the ultimate output of your algorithm; otherwise, we will blow your business into oblivion."
This is the type of vague, awful, impossible regulation that is focused on writing politically correct reports and which actually kills innovation. The UK's IoT rules might have had some edge cases that still needed to be worked out, but they were by and large technically-focused on real, serious security problems that had real, practical, technical solutions. Colorado, on the other hand, well, I honestly can't come up with words to describe how violently they've screwed the pooch.
I’m not going to go full devil’s advocate, here, because this looks like a textbook case of reactionary, something-ought-to-be-done legislation. But I’d like to hone in on one particular aspect.
Even without this law, I would expect AI decision making to fail disparate impact tests due to illegibility. Good luck proving business necessity!
But that cuts both ways. Once RLHF has beaten the racial slurs out of an AI, how are you going to prove that it was going for disparate treatment, even if it’s absolutely refusing to hire/promote/train a protected class?
No, seriously, I would like to see a standard which can distinguish between disparate treatment and impact. You can get a human on the stand and ask about intent, animus, whatever. I don’t think you can expect that to work for a computer program.
AI companies should be afraid of causing disparate treatment. It’s wrong, even when it makes more money. But an unregulated market doesn’t have much reason to care about right or wrong. Until we find a better way to draw the line, disparate impact is going to remain useful.
Without having thought about it super long, like letting myself gradually pick up examples over weeks/months, I can only think of a couple areas that have been able to resist a collapsing of disparate treatment and disparate impact. Credit scores and, currently hanging by a 6-3 thread, gerrymandering.
For gerrymandering, sigh. Honestly, it might just be that the Court is tired of these cases. They get stuck dealing with them over and over again, unlike most of the areas where the disparate treatment/impact distinction is collapsed.
For credit scoring, I think it's that there is soooo much money on the line from politically-powerful interests, plus a little historical "we've been using this for so long" factor. Would credit scoring have to fall under a strict interpretation of how these concepts work according to a radical (or even the otherwise dominant party line)? I think absolutely. Is the reason why it's been able to persist that you can get a human on the stand and ask about intent, animus, whatever? Not at all. Credit scores are an algorithm. An impersonal, just simple math, algorithm, with data in that may be subject to all the complaints people want to have about, "But if your data in is biased by a white supremacist patriarchy, then of course your algorithm is going to have racist and sexist disparate impact." Note that this Colorado law calls out that they're interested in:
No, the reason credit scores are still allowed is because too many connected people would stand to lose too much money if we let the collapse of disparate treatment/impact culminate entirely in the way that it seems to be going in nearly every other domain.
…and then tried to shuffle the boundaries so those 49% would never win again. No wonder they go for Biden.
I can’t believe that “we did it to make their votes worth less” is considered a legitimate defense. That’s obviously against the spirit of the Constitution and (in my opinion) ought to be illegal. Them’s the rules, I guess.
As for credit—I think we’ve got to distinguish between the different laws governing disparate impact.
First, you have the employment restrictions downstream of Title VII and Griggs. I think these are most likely to apply disparate impact, but also have the most explicit protections. We just don’t think about them as much because age and sex discrimination aren’t as politicized today. Construction, warehousing, meat packing…they’re obviously going to favor young men, but their “business necessity” precludes disparate impact. Cue commenters explaining how this is totally a feminist agenda. But I digress.
Gerrymandering isn’t covered by employment law. The opinion makes it clear that it’s a constitutional question. So I’m not surprised that disparate impact doesn’t come into it.
Neither of those laws apply to housing, which imports disparate impact via the Fair Housing Act. Like Title VII, that law is explicitly interested in racial justice.
Where does this leave credit scores insurers, and other actuarial pursuits? They’re certainly not mentioned in the Constitution. None of the titles of the 1964 CRA cover them. If there’s a later civil rights bill that does, I couldn’t find it. Instead, it appears that insurers are regulated by the states.
In other words, federal civil rights legislation doesn’t apply. There are guiderails on state regulations, but they date back to 1945 and use a more narrow definition of discrimination. Congress recognized that they shouldn’t mandate a product while also forcing it to be insolvent. Coming on the heels of the New Deal, this is pretty wild stuff!
Does credit get a similar exemption? Hell if I know. My point is that there’s a legal, intentional basis. Sometimes it’s not actually a corrupt bargain.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link