This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Apple has yet again betrayed the fact that "it's not technically feasible" is bullshit when it comes to government access
They said it concerning device access, and that was a lie. They said it concerning private cloud compute, and that was a lie. Hell, they said it concerning even the conceivability of secure backdoors, and that was probably a lie, too (not even getting into the fact that they definitely already have magic numbers that can overtly tell your phone to execute arbitrary code).
Apple has now introduced Enhanced Visual Search, technomagic which whizzes your photos (even the ones not in iCloud) off to Apple servers, using all sorts of mathematical goodies like homomorphic encryption to keep them private while allowing them to tag said photos with labels like, "What is this landmark in this photo?" It would be strictly easier to design a system that looks for stuff like child porn and alerts them.
Yes yes, you will rapidly object. False positives, false negatives. I grok filtering theory. Who will have the authority to do what with the information? Sure sure. Who will have to maintain the databases or filters that look for it? Yup, I hear ya. Those are not technical objections. Those are process objections.
I have never supported having the government involved in any of these things. I understand the significant nature of the tradeoffs at the societal level. But I have routinely said that the cry of, "This is just technologically/mathematically impossible," is total bullshit. It's just not true. Several other influential voices in the tech privacy world are coming around sort of slowly to this. They're seeing the deployment of these systems and saying how they're taken aback. How, well damn, if you can do that, then you probably can do this other stuff. But of course, doing the other stuff seems socially problematic, so they don't know how to feel.
It's actually easy to know how to feel. Just drop the lie that these sorts of things are technologically impossible. They are possible. But there are process tradeoffs and there are potentially huge liberty concerns that you can focus on. The more you continue to try to push the Noble Lie, the more likely you're just going to harm your own credibility in the long-term. Better to fight on the grounds of true facts with, "We don't want it," win or lose, than to prepare the grounds for a complete credibility crisis, such that when the time comes, no one is able to responsibly push back.
Well, there are some caveats there - if they are actually using homomorphic encryption to run the classifier, that means that Apple's servers do not at any point learn what the landmark is. If the goal is to report it to the FBI if a picture being sent has been labelled "child porn", accordingly, the phone would have to be wired up to report/send the image if the data it received, once decrypted, indicates that it was classified as such. How do you stop people from blocking this reporting functionality on their end? Adding additional user-unmuteable snitching logic to end-user devices comes with all sorts of legal, technological and security risks (and quickly puts you in a league with North Korean computing equipment that comes with daemons watermarking every document you touch, which they make it illegal to disable).
That being said, I see your argument at least insofar as the case for "it can't be done" is overstated and oversold, but I am not enough of an idealist to agree with this "just be truthful to the ruling classes and try to defend what you want on principle, the truth always wins in the end" thinking. I'm pessimistic about the prospect of a principled stand - we'll get the mandatory surveillance rectangle reporting on wrongthink eventually, because the powers-that-be really want it, and the majority of our fellow citizens probably already want it as well, or else the ruling classes will have all the opportunity on their side to manufacture the conditions that will make them want to, be it by propaganda, dissolving the cohesion of their opposition (note how effectively they split the tech anarchist scene into those who still want to keep the government out and those who think that the Nazis who want to keep the government out are the real danger) or creating real problems to which they are the solution (people want less government spying -> import scary foreigners into what to them is a scary foreign land -> old natives want more government spying to keep them safe from scary newcomers, newcomers want more government spying to keep them safe from racist natives). As far as I am concerned, the better choice at this point is just to lie and obstruct all the way. This buys time for some technical or societal deus ex machina solution to emerge, or else at least lets us spend a bigger fraction of our remaining time on this mortal coil out of bondage.
Are they currently claiming they do this today? My understanding of homomorphic encryption (admittedly a bit outside my wheelhouse) is that it's nowhere near as well-trod a space as, say, RSA. When I last looked, it was possible -- with a bunch of caveats -- to do simple things with a whole bunch of overhead, and certainly no equivalent of a NIST standard (if you trust those: say "Dual EC" with me) for it.
I didn't think the technology to do this well was ready for prime time today, but maybe I'm just a bit out of date. Do they have a white paper, or, better, a bunch of academic papers?
Yup. They even open sourced a library. They're citing the BFV papers from 2012.
Thanks. I suppose publishing a white paper at least opens them up to more serious scrutiny (which I've seen in serious-ish forums like HN). But my initial response is rather skeptical still, though. The encryption methods seem like they should be far more expensive than Apple is letting on [1], and they say they're using this for querying remote databases with encrypted queries. They're less clear on how these databases (for photographic landmarks, URLs, and such) are encrypted in a manner that actually hides the query from Apple. Are the encryption keys different per-device? If so, how do they avoid needing a separate database per device? And if not, it seems there's a lot of trust that they would be unable to figure out which rows matched.
I know Google's approach to similar issues has been focused more on device side ML models. Pixel phones support offline song recognition (I've noticed it's fairly limited to popular songs), and Google Translate can work (in a limited fashion) offline. Why does Apple need to do cloud-based POI recognition in photos? The whitepaper only shows 6 very well known landmarks, but it seems like it'd be easy enough (and secure!) to do this on-device. Given the known computational costs of FHE, it might even be better for battery life.
Oh, I would read it differently. I think they're letting on that it's pretty expensive, which is why they're doing all the mess with sharding and DP.
My understanding of BFV is that when the device does its keygen (unique for each query, I assume), it produces and then passes an "evaluation key" as part of the public key. One has to design the scheme so that there is sufficient expressivity in the set of evaluation functions that can take in the evaluation key and perform the desired operations on the encrypted message. I don't believe this involves completely encrypting the entire database from scratch using the public key every time; it just requires running the same operations on the same underlying database, but with the evaluation key in the operations. They call out that it's important that the BFV scheme has many of the operations they want, expressible in, shall we say "evaluation key parameterized form".
I think a lot of the work in library building is essentially building up a set of these "evaluation key parameterized" operations. You have to start with extremely simple operations and then build your way up to more usable tools that are composed of those simple operations.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link