Agenda — Autumn 2026 (Chicago)

Two 80-minute meetings per week, Tuesdays and Thursdays. Notes are reconstructed from the class recordings after each meeting; where a recording did not capture something (small-group discussion, report-backs) the entry says so rather than guessing.

Meeting 1 (Tue Sep 29)

• Introductions and setup
  • Course TA: Anna Lorimer, who researches in this area and is the first stop for project ideas. She will also give some lectures
  • Done in class: join Slack, create a private GitHub repository, give feamster read access (Settings, Collaborators), and fill out the intake form
• Course mechanics
  • The course page is a GitHub page embedded in Canvas. The only thing to take from Canvas is the book PDF; a newer copy will be posted
  • Eighteen class topics, taken in sequence, roughly one per meeting. No dates on the schedule, on purpose
  • Each topic has a discussion (breakout) and a hands-on activity. With 80 minutes we usually do one; the class can say which it prefers
  • Reading responses: one per week, due Tuesday 9 am (this first week only, Thursday). No summary. Just the questions you had after reading, a technical clarification you want, or something you want discussed. One sentence is enough. Graded done or not done. They set the agenda for that day’s discussion
  • Presentations: done in groups, on a topic of your choosing (a suggested list or a current event). Sign up in the linked sheet, three or four people per slot, starting in about two weeks. Link your slides in the sheet; past terms’ slides are there as examples
  • Participation: this is a discussion class. Show up and take part
  • Group research project: a proposal is due Friday October 23 (end of week 4): what you will do, with whom, why it is interesting, what has been done before, and what you need (data, tools) to finish. Talk to Anna before writing it. Past projects are on the linked YouTube playlist
    • If stuck for ideas: measurement projects (transparency reports, firewall, VPN, and Tor measurements; look ahead to chapter 5), or how AI models moderate prompts and outputs
  • Typical meeting: reading-response questions, a student presentation when one is scheduled, a short lecture, then a breakout or hands-on
  • Discussion norms: topics here touch contested issues. Respect differing views and approach them with curiosity. The instructor will not offer personal opinions on normative questions, so as not to put a thumb on the scale
• Course overview
  • History: the topic began around 2000 as the Great Firewall of China blocking web pages, then spread to other countries
  • Technical controls (weeks 1 to 3, the most technical part; no prerequisites): manipulating name lookups (DNS), connections (transport), and routes; slowing traffic (throttling); taking services offline (denial of service)
  • Platform controls: content moderation, flooding, propaganda, disinformation, personalization
  • Legal and economic controls: copyright, net neutrality, zero rating, demonetization
  • Measurement: tools for technical measurement; transparency reports
  • Responses: circumvention with VPNs and Tor
• Chapter 1 framing: from censorship to information control
  • Outright blocking is the crudest form. The broader idea is anything that makes information harder to get: a tax on access
  • Margaret Roberts, Censored: fear (deterring people from publishing or reading), friction (making access slow or costly), and flooding (burying information under other content)
  • Blocking is obvious and provokes a reaction. If the goal is to stay in power, friction and flooding can work better because people may never notice
  • Not only authoritarian states: by one widely cited estimate about two thirds of the world’s internet users are subject to some form of information control, and democracies order shutdowns too (India was the example)
• Breakout (short): age verification
  • Prompt: Age verification requirements for online platforms are legitimate and should not be considered a form of information control. Agree, disagree, or “it depends, on what?”
  • Context given: rules aimed at under-13 and under-16 users, Australia’s under-16 ban, and recent US litigation against Meta
  • Verification is imperfect: self-declared birthdays, guessing from posted content, facial age estimation that works poorly near 13 and 16. A wrong call removes a legitimate speaker, the same problem as a spam filter’s false positives
  • Points from the report-back: ID requirements raise privacy problems, especially for minors; users who are blocked go elsewhere
• For Thursday: a reading response, and read section 2.2.1 on DNS

Meeting 2 (Thu Oct 1)

• Reading-response discussion
  • Who decides what is acceptable to censor? The Cloudflare and Daily Stormer example from the reading. A student’s framing: the only practical difference between cases we approve of and cases we would not is that one company’s chief executive agreed. Is there a principled line, and should any single party hold it?
  • Platform, publisher, or pipe? Cloudflare is infrastructure, not a publisher. Platforms argue whichever role suits the moment: “just a platform” for liability (Section 230, later in the course), “entertainment” in a current tax dispute with the City of Chicago
  • Current events: European regulators, at the request of sports rights holders, have ordered the public resolvers run by Cloudflare, Google, and Cisco to stop resolving piracy sites
  • Consolidation cuts both ways. A few companies now resolve names for most users, which was not always so. Ideas for re-decentralizing came up (radio mesh networks, blockchain-based naming). Counterpoint raised by a student: Starlink in Iran is a highly consolidated provider that helps people get around state blocking, yet access then depends on that one company’s choices
  • Can ordinary variation be mistaken for manipulation? DNS answers differ by location by design. This led into the hands-on
• Hands-on: DNS lookups from different locations (about 10 minutes)
  • Look up google.com with dig, then again through a VPN in another location. The answers differ
• Discussion: how would you detect DNS manipulation, given that answers legitimately vary?
  • Check whether the answer is plausibly nearby, using latency (ping) or the path (traceroute); a “local” answer that is far away is suspicious
  • Check consistency within one geography. Load balancing and failover still cause some change
  • Check who owns the returned address (whois)
  • Fetch the content and compare; personalization makes this unreliable
  • Compare the TLS certificate presented
  • None is perfect. In general this is hard, which is why measurement studies of DNS manipulation are difficult
  • DNSSEC was set aside for time
• Encrypted DNS (DNS over HTTPS)
  • By default your operating system sends unencrypted lookups to a local resolver supplied by the network you joined. Anyone on the path can see and alter them
  • DNS over HTTPS moves lookups into the browser and encrypts them to a chosen provider: Cloudflare by default in Firefox, Google in Chrome. Introduced roughly ten years ago by Mozilla and Cloudflare; turning it on by default was controversial
  • Benefit: on-path parties can no longer see or tamper with lookups
  • Cost: you must trust the resolver operator, and resolution that used to be spread across many networks is concentrated in a few companies, a new chokepoint
  • Usability: only two students had heard of it. The settings are hard to interpret, and an attempt in class to point the browser at a different public resolver did not work. Left as an exercise
• Breakout: is centralized encrypted DNS a net gain or a net loss for user autonomy?
  • For: encryption, and cases like Turkey where a large public resolver helped users get around local blocking
  • Against: one operator can refuse to resolve a name for everyone, and most users would never know why a page failed to load
  • The recording did not capture the group discussion or report-back

Meeting 3 (Tue Oct 6)

(The recording begins partway into the reading-response recap.)

• Reading-response themes
  • The same mechanism serves protection and censorship: a firewall at a school, a hospital, or a national border looks identical to the user whose connection fails. How do we disambiguate protection from censorship?
  • Encryption removes the fine-grained signals a censor used to have, so the response is coarser: block all encrypted traffic, or a whole site rather than one page. Is that a win?
  • As traffic concentrates in fewer content delivery networks and hosting providers, each chokepoint becomes more powerful
  • The arms race: if the censor can always find another way, is it worth raising the cost? This is where the course’s wider framing returns: when blocking gets expensive, control shifts to friction, flooding, and platforms
  • About half the class had responses in by Tuesday; updates before Thursday are welcome and still shape the discussion
• TCP mechanics (for the questions asked)
  • The three-way handshake: SYN (synchronize), SYN-ACK, ACK. Drawn as two machines, client on the left, server on the right
  • It is the Two Generals problem: nobody ever learns whether the final ACK arrived. The sender simply starts sending data. Teardown (FIN, acknowledgments in both directions) has the same property
  • A reset (RST) aborts a connection or refuses one. Normal uses: a SYN sent to a port with no listener gets a reset (“go away”); a killed process; any fast teardown
  • Forged resets: the Great Firewall watches for a SYN to a blocked destination and sends a reset to the client, which aborts regardless of what the server later replies
  • “Ignoring the Great Firewall of China” (Cambridge, 2006): if the client simply ignores resets, the connection proceeds. It worked because the firewall was stateless, sending a reset and forgetting the connection. The firewall later reset both ends
  • Injecting a duplicate or a reset requires being on the path: a router or other equipment between client and server, which in practice means a government or an ISP. The forged packet must carry a plausible sequence number; sequence numbers are 32-bit, visible in the clear (TLS does not encrypt TCP headers), and because a reset is accepted anywhere inside the receiver’s window they can be brute-forced if necessary (correction: 16-bit was said in class)
  • The economics: an on-path censor only has to win a race, sending its packet before the real reply, so the manipulation is cheap once the vantage point exists
• Why transport-layer manipulation is becoming less attractive
  • Encryption hides content, and shared hosting (one cloud or CDN address serving many sites) means an IP address no longer identifies a site
  • Censors answer with website fingerprinting: even encrypted, a page has a characteristic number of objects and sizes and a characteristic timing, which together act as a fingerprint of the site being visited
  • Questions from the room: oblivious DNS does not help here (it protects the lookup, not the transfer sizes); padding traffic with extra data does help and is the standard defense, with a large research literature behind it
  • Net: on-path manipulation needs a vantage point plus fingerprinting; friction, flooding, and platform-level controls are easier. Techniques for detecting TCP manipulation return in the measurement unit
• Breakout B (short): is the cat-and-mouse game winnable?
  • Prep-read tour: QUIC, a newer transport developed by Google and used by Chrome, encrypts more of the connection setup and can split a connection across paths, which raises new censorship questions. Tor’s pluggable transports exist to protect the initial connection to the network, which is the sensitive moment for any VPN-like tool; each transport (obfs4, then Snowflake) gets fingerprinted in turn, and Russia recently blocked Snowflake. VPNs move the trust problem to the provider
  • Report-back: censors have many tools and take the path of least resistance, which can be economic or legal rather than technical. The asymmetry runs both ways: a circumventing user must hide every aspect of their traffic while the censor needs to spot one anomaly, but the censor must close every channel while the user needs one that works
  • Can traffic evade increasingly good machine-learning detection? Open question; flagged as a project direction (pit detection models against generative evasion). The role of AI in this race was deferred
• DNSSEC (secure DNS, as distinct from encrypted DNS)
  • The resolver walks the hierarchy: root, then the .edu servers, then the uchicago.edu servers; each step is a referral
  • With DNSSEC each referral is signed. A signature is made with a private key and checked with the matching public key; the referral carries the next zone’s key, signed by the zone above it. This is integrity, not confidentiality: encrypted DNS hides queries, DNSSEC proves answers were not altered
  • The chain starts at the root key, which ships with the operating system or software. Same root-of-trust question as certificates: you have to start by trusting something
  • What it defends against: an on-path forged answer cannot carry a valid signature, so it is rejected. What it does not: a compromised root key, or a bad key added to the root store. Keys have been added to and removed from operating-system root stores over the years. This is a quieter form of consolidation: whoever holds a root key controls everything below it
  • Demo: the operating system’s keychain lists the root certificates it trusts; most users never look
• Hands-on: dig with DNSSEC
  • dig +dnssec shows the signature records (RRSIG). +trace shows the full iteration from the root, with each referral, its signature, and a DS record pointing at the next zone’s key
  • UChicago’s domain is now signed, which it was not a year ago; so is a major news site. Deployment is far broader than when the exercise was written. Task: find a domain that is not signed
• Parked for Thursday
  • Why DNS manipulation is so much more common than routing (BGP) manipulation, and the other BGP questions from this week’s responses
• Logistics
  • Presentations start in about two weeks; sign up if you have not

Meeting 4 (Thu Oct 8)

• Housekeeping
  • Most week-2 responses are in; the few who have not should push them. Sign up for presentations, which start the week after next. Project proposals are due Friday October 23; Anna will help give feedback
  • Give Anna Lorimer read access to your response repository as well, if she is not already a collaborator
• Reading-response follow-ups
  • Machine learning in the arms race (asked several times): for protocol-level interference there is no obvious advantage to machine learning, as far as the instructor is aware. The question becomes much more pertinent at content moderation and platform decisions, later in the course
  • Why DNS rather than BGP? DNS is easier to manipulate: operate a resolver and get people to use it, then return the wrong answer. It is also harder to detect, because DNS answers vary naturally. Routing manipulation is a sledgehammer: brute force, and it works if it works. For a complete shutdown (Egypt) withdrawing routes is the easier tool, and India uses it more often than most. In general DNS is manipulated far more than routing
• Lecture: routing and route hijacks (book 2.2.3)
  • The internet is a collection of independently operated networks, autonomous systems. BGP, the Border Gateway Protocol, is simply the routing protocol those networks use to talk to one another. Directions one hop at a time: a network gets your traffic as far as its border, then the next network takes over. Not a full route the way a map app gives one; the reason is scalability (nobody can run a shortest-path computation over the whole internet), so BGP adds a layer of abstraction
  • Security is very limited. Efforts to secure routing have largely failed except for the Resource PKI (RPKI). Recommended easy read: Sharon Goldberg, “Why Is It Taking So Long to Secure Internet Routing?” (Communications of the ACM, 2014), the source of the diagram shown
  • Hijacks. Any autonomous system can advertise any block of addresses to its neighbors, and routers mostly believe what they hear and pass it on. Mistakes cause outages (at least one of the recent large cloud-provider outages was a routing misconfiguration); malicious parties can do the same deliberately
  • Prefix arithmetic, on the board. An address is 32 bits, four 8-bit numbers. 128.135.24.0/24 means the first 24 bits are fixed and the last 8 are a wildcard: 2^8 = 256 addresses. A /23 fixes 23 bits and covers 2^9 = 512 addresses, so a /24 is more specific (a longer prefix) than a /23. The table shows overlapping prefixes like these
  • A student asked: if several routers advertise the same addresses, which one does a router pick? The most specific (longest) prefix wins. Among routes of equal length a set of rules decides, one of the higher-priority ones being the fewest autonomous systems on the path, which is correlated with distance but need not be
  • Fighting back. A hijack is not something a user sees easily: a withdrawn route is obvious (no internet), but traffic quietly detouring through another country may go unnoticed. Step one is detection: a network can watch public route collectors for someone else advertising its addresses. Step two: you cannot order the hijacker to stop, but you can split your own prefix in two and advertise the halves, which are more specific and so win. This works only up to a point: if everyone advertised ever-longer prefixes the table would grow exponentially, so many routers filter or ignore prefixes longer than /24
  • Questions from the room. Could you advertise half the internet with a /2? You could try; it would lose to every more specific route. Is there a law against hijacking? The instructor knows of no prosecution or case law; there are legal theories but nothing tested. Who can actually do this? You need a router that another network’s router is willing to believe. The barrier is not in the protocol but in the setup: convincing an operator to connect to you and speak BGP. Small ISPs enter such arrangements all the time (a university routing for a local library, for example), so it is not far-fetched. The 1997 AS 7007 incident from the reading was caused by a very small ISP
  • A student asked: if BGP only gives step-by-step directions, how does anything know where to send traffic? Drawn on the board from a real table entry: UChicago’s 128.135.0.0/16, reached through an intermediate provider and then Internet2, the US academic research backbone. Each network on the path only knows how to get the packet to the next border; whois turns the autonomous-system numbers into names
  • A student asked: why does withdrawing a route work; can’t routers cache the old one? In theory they could and sometimes do, but a cached route cannot tell a deliberate withdrawal from a genuine failure, and reacting to failures is the whole point. The course theme again: distinguishing a legitimate failure or glitch from an intentional act, whether a route, a DNS answer, a moderation decision, or (Tuesday’s topic) slow traffic
• Hands-on: RouteViews (activity, about 20 minutes)
  • telnet route-views.chicago.routeviews.org, then show ip bgp <address> for UChicago and YouTube; show ip bgp alone dumps the whole table, which shows the overlapping prefixes. RouteViews is a public “telescope” of route collectors; the project also publishes historical routing data, a rich source for a project on routing manipulation
  • The old list of collectors on routeviews.org had disappeared; the activity was updated during class with the current list and a note to brew install telnet on recent macOS
  • The activity’s discussion questions (how a censor could use hijacks, how to detect or prevent them) were covered by the lecture; the class had no further reactions to report
• Not covered: RPKI in detail, and the question from the responses about whether securing BGP concentrates power. Stated briefly: a centralized routing PKI means the root could revoke an autonomous system’s certificate, after which other routers would drop its routes. That is a new problem that did not exist before RPKI. Breakout A (should governments mandate RPKI) was not done
• Breakout: should an ISP refuse a government shutdown order? (breakout, Breakout B; about 10 minutes)
  • Framing: the same question will come back for platforms (China has asked Google to censor search results, and Google has gone back and forth on it). For ISPs: ordered to remove prefixes or shut down, should they comply? What can they do, given that refusing may cost a license or a fine? No ISP is known to have tried, or if one did, the discussions were never public. A student’s earlier example of Starlink was raised as a provider that reaches into a country from outside and may not be bound the same way
  • Two variants to discuss: a complete shutdown, and the greyer case of a single prefix that is harder to notice
  • The recording did not capture the small-group discussion or the report-back
• For Tuesday: week-3 reading response by 9 am; read 2.3.1 on throttling

Meeting 5 (Tue Oct 13, on Zoom)

• Housekeeping
  • Class was held on Zoom; a few students joined from the classroom. Most week-3 responses are in. Project proposals are due Friday October 23
  • Presentation sign-ups: nobody signed up for today’s slot (filtering and throttling), so student presentations start Tuesday October 20 with personalization and filter bubbles. Anyone not yet signed up should do so now
  • The classroom camera will be set up on Thursday so that any future Zoom session can show the room. Thursday’s class is in person; there will probably be one more Zoom session this quarter
• Framing: degrading instead of blocking (book 2.3.1)
  • Everything so far has been outright disruption: poisoned DNS answers, reset TCP connections, manipulated routes. Today’s topic is subtler: make a service slow rather than unreachable. Many users give up, and there is no hard evidence that anything was blocked, because a deliberate slowdown is hard to tell from poor network conditions
  • Two places to degrade traffic, at opposite ends of the path. On the user’s side, throttling: a shaper slows the sender, and it can be very selective (one application, part of one application, certain sites) and made to look like an ordinary slow network. On the service’s side, a denial-of-service attack overwhelms a finite resource so the server itself looks down or slow. Example: China’s attack on GitHub, which hosted circumvention tools promoted by GreatFire.org. Both look like performance problems, both disrupt availability, and neither leaves a clean pointer to the censor, which is the deniability the censor wants
  • Throttling (rate limiting) is a legitimate network-management tool that keeps one user or application from hogging shared capacity; the same mechanism becomes censorship depending on the target and intent. Molly Roberts calls this friction: raising the cost of reaching content until users quit
• How a traffic shaper works
  • Two steps: identify the traffic (which application, which destination), then shape it. Traffic is sorted into queues by service (video in one, everything else in another) and the shaper controls how fast each queue drains
  • Identification is an arms race with encryption. Three methods and their weaknesses: watch DNS lookups (for Netflix, say) and throttle the addresses that come back, which over-throttles when addresses are shared; read the domain name in the TLS handshake (the Server Name Indication), which Encrypted ClientHello now hides; or inspect packet sizes, timings, and signatures (deep packet inspection), which is costly and degrades as more traffic is encrypted. Breaking identification is therefore one of the better defenses against throttling
  • Encrypted ClientHello is deployed in current browsers, and censors have responded by blocking the TLS versions that carry it. Lesson: encrypting the identifier does not end the arms race; the censor targets whatever new pattern the defense creates
• What the shaper actually does to your traffic (the question nearly half the class asked in some form)
  • The leaky bucket and token bucket are scheduling abstractions, not devices that handle packets. A leaky bucket drips at a set rate; each drip tells the router to take one packet off the corresponding queue and send it, so the drip rate is the sending rate. There is typically one bucket per queue or per rate you want to enforce, and different queues can drain at different rates. A token bucket is a bank balance: tokens accrue at a set rate, and you can pull packets off the queue as long as you have tokens to spend, which allows bursts. Once the tokens are gone, nothing is sent and the buffer fills
  • What a user sees, in order. First, higher delay: the device serves the queue slowly, so packets wait in line. Then, if packets arrive faster than the queue drains, the queue fills and the device drops packets. Dropped packets mean lost data, and senders interpret loss as congestion and slow themselves down (TCP congestion control). The shaper itself drops nothing; it serves the queue slowly, the queue overflows, and the sender’s own protocol finishes the job
  • Can a censor make this indistinguishable from congestion? Largely yes: a shaped service plan from your ISP looks exactly like this, and a congested network also produces higher delay and drops
  • Bufferbloat (asked in a response, to be defined in the book). Buffers sit in routers and exist because traffic is stochastic and bursty: everyone in a room may send at once or at different times, and the buffer absorbs that variation. Buffers that are too large produce huge delays with no drops, so nobody slows down; drops are the signal to slow down, and newer transport protocols also treat rising delay as a signal
  • Narrowly targeted throttling is harder to detect than slowing everyone down: the more focused and the more variable it is, the harder it is to see
• Reading-response follow-ups
  • Why is the hostname in the clear in HTTPS? One address hosts many sites (content delivery networks), so the server needs the name before it can choose which certificate to present. Chicken and egg: without the name there is no handshake. Encrypted ClientHello fixes it by publishing a public key in DNS so the client can encrypt the first message, which means ECH depends on DNS. Fetching the ECH configuration without encrypted DNS: not really; encrypted DNS goes to a handful of well-known resolvers, and a country-level censor’s answer (China’s) is simply to own the DNS, which blocks a proper ECH deployment at the bootstrap
  • Does more encryption help or hurt? Both: encrypted DNS and ECH make identification harder, and censors respond by blocking TLS or ECH outright. Left as a discussion question
  • A student’s note about the book’s description of QUIC was right; the text will be fixed
  • VPNs. Throttling does not stop traffic from hiding in a VPN; a VPN raises the bar for identification, which is one more reason to use one. Why not block all VPNs? Students offered: legitimate uses and collateral damage, and businesses that need them; blocking is coarse, all-or-nothing, and visible, while fine-grained interference is hard to detect. Related: a Roberts paper found that blocking certain websites increased VPN installation and, in the end, access to information
  • Evidence and deniability (several responses): what evidence would show deliberate throttling, and how to tell legitimate rate limiting from censorship. Today’s hands-on tries to detect it, and the upshot is that detection does not work reliably. If an ISP knowingly leaves a peering link congested, when does a business dispute become information control, or are they the same thing? Put to the class as a discussion question
  • Laws. Net neutrality rules have been vacated in the US; California has a state law and the EU’s Open Internet Regulation prohibits per-application throttling. Every one of these restricts what an ISP may do; none restricts a government acting through an ISP, and all are silent on content delivery networks and caching, which is where much performance optimization or degradation happens
  • Who should be responsible for detection? Tabled until the measurement unit
  • Does social media make throttling easier to detect? In a way, yes. Shown live: Downdetector, where crowd reports corroborate an outage (GitHub was showing user-reported problems during class). The censorship measurement tools later in the course, OONI and Wehe, are crowdsourced for this reason
  • Could I detect Netflix throttling from a packet capture at home? You could detect degraded performance, but not whether the cause is the censor, your Wi-Fi, or your ISP; that is where crowdsourced measurement helps
  • Is throttling ever good, for example to keep children off content? Left for debate. The instructor’s only comment: if the goal is to keep children away from content, blocking is more effective than slowing it, which leaves open where the line between parental controls and censorship lies
  • Is throttling inherently bad? Cloudflare does it. This is the network-management-versus-information-control problem: the mechanism is necessary to run a network well, and turned up too high it is censorship. Captchas are a form of rate limiting on the user, friction in the name of denial-of-service defense
  • Who is hurt most, and does sophisticated censorship create inequality between those who know about and can pay for VPNs and those who cannot? Probably yes. The instructor knows of no rigorous research on it and would like to see a paper: how do sophistication, money, and access to knowledge create inequities in access to information
  • If throttling never blocks, is it effective; can’t people just be patient? Left open; friction is friction
  • What if an AI agent fetches the content for you? If an agent gathers your news overnight while you cook or sleep, web latency matters less. Raised as a research question: could agents be a way of fighting back against throttling, since the fast user interface is no longer the bottleneck
  • From the chat: Miranda Fricker’s book Epistemic Injustice, on injustice in the distribution of and contribution to knowledge, read into the record as relevant to this discussion
  • Famous throttling events: Comcast and BitTorrent (2007); the Netflix and Comcast peering dispute (2013–14); Verizon throttling a California fire department’s data plan during a 2018 wildfire; Iran’s repeated slowdowns during protests; Russia’s throttling of Twitter (2021) and later YouTube and WhatsApp, which are in the slide deck. Many of these are commercial, as a student also pointed out: how much throttling is commercial rather than governmental
• Hands-on: Wehe (activity, about 15 minutes in Zoom groups of about five; the students in the classroom worked as their own group)
  • Install the Wehe app, run it against several applications on your own network, and compare. The expected result was a circus of inconsistent findings, and that is what happened: reports back were inconclusive, with no differentiation detected on any app; the instructor found nothing either
  • The tool seems to get less reliable every year: some users see an app flagged and others do not, and the mechanism (read the Wehe page) is not statistically sound. Where it has shown differentiation in the past is on cellular networks, which do throttle some applications and are fairly open about it as “network management,” whether or not that is anti-competitive. Either throttling is harder to detect now, or there is less of it, or the app no longer works; this hands-on may be retired
  • From the chat: would testing through a VPN from a more restricted country have shown more? Probably; it would be interesting to see where VPNs themselves are blocked
• Closing discussion (about five minutes; Breakout A was not done for lack of time)
  • Is throttling worse than blocking? One student said yes. Another made the sharper observation that “worse” is normative, but it is true that throttling is more insidious because it is harder to detect and to disambiguate from ordinary failure
  • From the chat: could AI agents themselves be throttled or blocked, and if everyone uses agents, does censorship simply shift toward them? Yes, probably. Access to agents, and agents performing at different speeds, become the new axis. A current PhD project asks whether people in different regions get different access to information through agents and retrieval-augmented generation, in both content and speed. Completely open territory for a research paper or term project. Also from the chat: API token limits are a form of throttling, and agents are expensive
  • From the chat: what is morally worse, throttling or blocking, depends on which is more effective at restricting information, which needs data. Agents and token limits throw a wrench into that comparison
• Not covered: the denial-of-service half of the deck (Thursday); Breakout A on plausibly deniable censorship. No exam flags were given
• For Thursday (in person): denial of service (2.3.2). Class will open with a breakout on the questions left hanging today: agents versus throttling, and throttling versus blocking

Meeting 6 (Thu Oct 15)

The recording begins at about 4:05 pm, partway through the report-back of the opening breakout. Housekeeping and the breakout itself were not captured; the dates announced are the ones posted in Slack.

• Housekeeping (not on the recording): week-4 reading response (book 3.1–3.4) due Tuesday October 20 at 9 am; project proposals due Friday October 23; student presentations begin Tuesday
• Opening breakout, report-back: can AI agents defeat throttling?
  • It depends on what the agent does. A summarization agent (the model reads the news and summarizes it) raises hallucination and even malicious-summarization worries. A prefetching agent that pulls the pages you read into a local cache is a different thing: the set of sites a person actually consumes is small, storage is cheap, and an agent could fetch that slice of the web overnight. Throttling a site the user never visits live loses much of its effect
  • A student reported that ChatGPT describes its retrieval as using its own caches, so a throttled site may already be cached either locally or at the model provider. The instructor’s open question: if friction stops working, does censorship swing back toward blocking, after the Roberts argument that friction beats blocking?
  • Paywalls as a related form of access control. Hard paywalls (Bloomberg, legal news services) block everything behind a login. Soft, client-side paywalls (New York Times, Wall Street Journal) usually return the whole article and hide it with JavaScript, so a scraper or an agent gets the full text. Publishers built them that way on purpose so search engines would index the articles and send paying readers; agents that read straight through the paywall changed the calculus. Flagged as a good research-project area
  • A student argued that an agent on your own machine uses your IP address and speaks the same HTTP, so it is throttled exactly as you are; a cloud agent uses a different address, but the censor can throttle that the way it throttles VPN providers. The instructor agreed agents are no less likely to be throttled, perhaps more, but noted that if the interaction is asynchronous (ask for a digest, come back later) the delay matters less
• Lecture: Denial of service (book 2.3.2; this week’s “crossover episode” with the Wednesday security class, kept short because the reading covers the mechanics)
  • Throttling slows things down at the client; denial of service slows the service for everybody. The focus here is denial of service launched by censors
  • Ways to deny service: saturate network capacity, or overload the server by exhausting its memory or CPU. A distributed attack recruits thousands of machines each sending a little traffic, focused on one victim like a magnifying glass focusing sunlight
  • Amplification: a small request produces a large reply, the classic case being a small DNS query to a resolver. Reflection: spoof the source address so the large reply lands on the victim instead of the attacker. Analogy: mailing a business a postcard with your enemy’s return address so they receive the catalogs. Multiply by thousands of compromised machines
  • Reading-response questions answered in class
    • Is there a packet type that cannot cause denial of service? No. Anything with a finite resource (network capacity, CPU, memory, storage) can be exhausted. Example: a firewall installed to stop bad traffic becomes the target; exhaust its memory and everything behind it goes dark
    • Can a defense be designed so the defender never does more work than the attacker? That is the idea behind nearly every defense
    • Can packet captures tell a DDoS from ordinary congestion? Very hard
    • Has anyone run denial of service against a censor, given that deep packet inspection is expensive? The firewall point applies: an on-path censorship box that falls over takes down everyone behind it, so it is not obviously useful. The instructor called his own answer glib and the question a good one. A student asked whether a foreign adversary could cut China off by attacking the Great Firewall; in principle maybe, but how it is provisioned is not well understood
    • How hard is it to verify the source of traffic? An attacker who never needs the reply can lie about the source address. Egress filtering: the router at the edge of a network such as UChicago’s drops outgoing packets whose source is not in its own address block. It relies on every network being a good citizen and costs money, so many do not do it, and once spoofed traffic leaves the leaf network it is very hard to filter. Ingress filtering (drop incoming traffic that claims to come from our own addresses) is somewhat more effective but does not protect against other spoofed addresses
    • A student asked whether any regulation requires such filtering. The instructor knows of none from governments; it exists as a published best common practice
  • The Great Cannon (about ten years ago, named by the researchers who documented it). The Great Firewall’s usual move is to sniff traffic and inject TCP resets. The Great Cannon instead injected JavaScript into responses to users outside China who loaded content from Baidu; the injected script made those browsers send requests to GitHub, which hosted circumvention projects, and to GreatFire.org. The censor recruited outside browsers as its attack machines
  • Not covered: SYN floods and SYN cookies, Mirai and the Dyn outage, and the defenses slides. The security-course lecture covers the first two for students in both courses
• Project Galileo
  • The standard defense against volumetric attack is more capacity: host the site in more places or with more bandwidth, which costs money. Hypothetical: a university site such as a free-expression forum is a plausible target and is not provisioned for terabits per second
  • Cloudflare, a content delivery network, started Project Galileo in 2014: if Cloudflare decides a site is worthy and it is under attack, it gets denial-of-service protection for free. The question is who decides
  • Reading-response questions put to the class: who decides which sites get defended; the risk of relying on a few big providers; how Cloudflare decides who gets protection free; whether Galileo protects groups outside the mainstream (the instructor thinks yes, and the hands-on asks you to check); whether defense that requires Cloudflare-scale capacity is in tension with free speech
  • Hands-on, about six minutes in groups (activity): who is protected, are they mainstream, what it takes to join, and whether a personal website could. Imagine being the Cloudflare employee making the call
  • Report-back (90 seconds, largely not captured): one student suggested a tiered approach where those who can pay do and others are protected free. The instructor said the discussion is to be continued
• Not reached: the whole-class discussion on DDoS protection as a common-carrier obligation (Breakout B)
• Next: flooding, propaganda, and disinformation (book 3.1–3.4), with the first student presentation

Meeting 7 (Tue Oct 20)

The recording begins at about 3:41 pm, partway through housekeeping, and captures the reading-response discussion and the student presentation up to its discussion questions (about 4:11 pm). The rest of the class, including the presentation’s discussion and the closing block, was group conversation the recorder did not pick up.

• Housekeeping
  • Project proposals are due Friday October 23. A group may submit from one shared group repository or from one member’s course repository; a group that uses a shared repository should add a pointer to it in each member’s private course repository so the staff can find it (posted in Slack: add the instructor and the TA as collaborators)
  • The chapter-3 section numbers on the schedule were corrected to match the current book
  • Week-5 reading response (book 3.6–3.7, moderation and personalization) is due Tuesday October 27 at 9 am (posted in Slack; the agenda post had first said Thursday, which was wrong)
  • The instructor announced that the end of class would be the sentiment-analysis hands-on (activity) instead of the posted breakout, then switched back to the breakout after the presentation covered the hands-on’s ground; the hands-on moves to the content-moderation week
• Reading-response discussion (three themes from the week-4 responses)
  • Is flooding censorship at all? Several responses asked whether something that works by adding speech rather than removing it can be called censorship. The instructor agreed it is not censorship in the strict sense; it is a form of distraction or control of attention, which is why the book’s introduction hedges toward the term “information control,” a term that is not widely accepted either
  • Flooding vs. organic collective speech. If thousands of people organically post on one topic and drown out other information, how is that distinguishable from a coordinated flooding campaign? The instructor did not have a clean answer; the effects may be similar. A student pointed out that the algorithm rewards engagement, so creators write in response to what the algorithm is amplifying: the algorithm tilts, humans latch on. The instructor called this an astute observation, said they have not seen a study of creator behavior in response to ranking, and flagged it as a possible (if hard) project topic
  • Dead Internet theory. A student explained the two versions: the original conspiracy version (nothing you have seen online since 2016 is real) and the softer version (the large majority of accounts you encounter are bots, with perhaps one in ten a real person, so much of what you observe is bots talking to bots). A past student project on the topic is in the course playlist
  • AI as the arbiter of information. Many responses noted that chatbots refuse certain questions in opaque ways and asked whether an AI system or search engine should have to explain which policy caused a refusal; nobody outside the companies knows the system prompts, the post-training, or the alignment choices. Responses also observed that models disagree with each other on what is allowed, and one noted that humans given the same content disagree too. The recurring question was whether platforms should be required to disclose which models they use and how those models are set up. The instructor tabled this for after the presentation and said it pairs with the sentiment-analysis hands-on
  • Tiered AI access. One response asked whether paid models allow or answer things the free tiers do not, and what information gap that opens. Deferred to the later lecture on tiered pricing, zero rating, and net neutrality, where it belongs with payment for access to information
  • AI agents and flooding. As more of the web is read by agents rather than people, does flooding lose its lever, or does it shift to influencing what the agent summarizes? Same question the class asked about throttling last week. A student cited a figure of 275 million weekly commits to GitHub, up about fourteen times year over year, much of it attributed to public pages and repositories; the instructor added that their own commit rate has gone up by one or two orders of magnitude in a year
• Student presentation: personalization and filter bubbles (first presentation of the term; slides are linked from the sign-up sheet)
  • Personalization is a continuous loop: collect profile data (age, gender, location) and behavioral data (searches, clicks, engagement); predict what content is most relevant and engaging; push it; measure engagement; re-rank and repeat
  • Personalization is the mechanism; a filter bubble is a possible outcome, in which a person sees a narrow range of information, content that reinforces existing beliefs keeps arriving, and opposing views become rare or absent
  • History: 1990s information filtering (the 1992 Tapestry system, which let users filter documents using other users’ annotations; the mid-1990s MIT Media Lab idea of a personalized daily newspaper; Amazon’s 1998 cookie-based recommendations); 2000s mass personalization by Google, Netflix, and Amazon, including Google’s personalized search for all users in 2009; Eli Pariser’s 2011 book and talk that popularized “filter bubble”; TikTok’s 2016 shift to discovery driven by what you watch rather than whom you follow; and 2020s generative-AI personalization. What changed: the goal moved from managing information overload with explicit feedback to shaping what people encounter in order to optimize engagement metrics, pushing content people never sought out and now generating it
  • Open debates the presenters listed: whether filter bubbles are overstated; whether personalization causes polarization and radicalization or only reflects divisions already in society; whether it empowers users or undermines their agency; whether platforms should be regulated to prevent bubbles; and whether platforms should show people content they do not want to see
  • Class vote by show of hands: who is more responsible for what you see, the algorithm or the user (the tally was not captured)
  • Case study: the 2018–2019 narrative that YouTube’s recommender pushed casual viewers toward extreme content
    • A 2020 audit of “radicalization pathways” grouped channels into mainstream media, the intellectual dark web, alt-lite, and alt-right, used commenting on a channel’s videos as a proxy for exposure, and simulated users following successive recommendations. About 12 percent of users who commented only on intellectual-dark-web videos in 2017 commented on alt-right videos by 2018; roughly 40 percent of 2018 alt-right commenters could be traced to cohorts that had commented only on the two milder groups; about 4 percent of random walks starting from alt-lite channels and 1.5 percent starting from the intellectual dark web reached alt-right channels within five recommendation steps. Pathways from milder to more extreme content exist and recommendations help people find extremist channels, but commenting is an imperfect proxy and the causal role of the recommender is hard to establish
    • YouTube’s 2019 change: recommend fewer borderline and misinformation videos (miracle cures, flat earth), which remain searchable; external reviewers rate videos against the guidelines and classifiers are trained on those ratings. YouTube reported a 70 percent drop in watch time of borderline content recommended from channels the viewer had not subscribed to
    • A later counterfactual-bot study: pairs of bots started from the same real users’ watch histories, one continuing to follow what the human actually watched and the other following YouTube’s recommendations. The bots following recommendations watched less far-right content than the bots replaying human choices. When a user switched from far-right to moderate videos, far-right recommendations faded after about thirty videos, while the human’s own choices changed more slowly. The presenters’ conclusion: user choice mattered more than the recommender for who ended up on far-right content, and on average the algorithm recommended less extreme content than people chose for themselves
    • Discussion questions the presenters posed: if people mostly choose this content themselves, does that let platforms off the hook; and how can one tell whether an article is organic or planted. One student observed that the selection bias runs both ways: creators make what they think the algorithm will favor, so content the algorithm does not favor is never produced to be shown, much as search results fill with AI-generated articles. The remaining discussion was not captured
  • The instructor noted that building discussion into a presentation was a good innovation that other groups are welcome to copy
• Not covered: the flooding and propaganda lecture (book 3.1–3.4: fear, friction, and flooding; the 2014 Instagram block; the 50-cent army; behavioral signatures of campaigns; exposure vs. persuasion) was not reached. The closing block was Breakout A, amplification vs. astroturf; the group discussion and any report-back were not captured
• Next: disinformation (book 3.5), with the flooding and propaganda material to be picked up

Meeting 8 (Thu Oct 22)

The recording begins at about 3:40 pm, partway through housekeeping, and captures the lecture and the start of the hands-on (about 4:32 pm). It stops at about 4:39 pm, while students were working in pairs, and a second short recording picks up the last minute of the report-back; most of the report-back was not captured.

• Housekeeping
  • Project proposals are due Friday October 23. The staff will pull them from the repositories and let groups know if anything is missing. Students still looking for a group were asked to say so in class so they could be matched; one student did
  • Week-5 reading response (book 3.6–3.7, moderation and personalization) is due Tuesday October 27 at 9 am
  • The third student presentation, on content moderation, is Tuesday October 27. The LLM sentiment-analysis hands-on (activity) moves to that day as well, since it fits the presentation’s topic; the presenters may build on it, run it, or do something else alongside it. The hands-on on filter bubbles was effectively covered by Tuesday’s presentation, so the syllabus’s hands-on column shifts accordingly
  • Anthropic once listed content moderation among the published use cases for its models; the instructor checked in class and the page is still up, which is the starting point for that hands-on
• Carried from Tuesday (book 3.1–3.4)
  • Why a regime might prefer flooding to blocking: the Roberts study of China’s 2014 Instagram block found that blocking caused people to install VPNs, so it backfired. Flooding leaves no hard evidence of censorship, so there is less likelihood of backlash, and there is nothing to circumvent: a block can be circumvented, flooding cannot
  • Tuesday’s discussion of the 50-cent party, including students’ first-hand accounts, was recalled. An open question from the responses: does the 50-cent party still exist, and why employ people at all when an LLM could generate the posts? The instructor did not know and invited students to dig into it
  • The remaining flooding slides (the behavioral signatures of a campaign) were not reached
• Reading-response discussion (week-4 responses on book 3.5)
  • Can LLMs detect satire? A site like The Onion is identifiable from the source alone, but a tweet or a blog post is not. Satire without context is the long-standing Achilles heel of automated classifiers, and humans often cannot tell satire or parody either, so it should not be surprising that models struggle
  • LLMs and the cost of flooding. Several responses observed that LLMs make it cheap to generate fake user content and to impersonate a writing style. The instructor agreed without reservation that LLMs bring the cost of flooding down; what to do about it is open
  • Is disinformation illegal? Taken up in the lecture (below)
  • Is the problem removing false content fast, or going after the systems and people behind campaigns? Probably both, depending on context. The bot farms from Tuesday that pump fake streams on Spotify are what used to be called click fraud, where the goal is clearly to catch and punish the perpetrator. For something like COVID disinformation the concern is also preventing spread. Students argued both sides: prevention before harm is done versus punishment after the fact. One student described a common spread pattern: content reaches someone vulnerable to it, who shares it with family and friends, so even if online spread were stopped the message travels offline and re-enters the network from people who are neither governments nor bots. The instructor connected this to social proof: people are more likely to believe something from a close friend or trusted source, which shapes how and why information spreads
  • Accountability. A student raised the 2003 invasion of Iraq, justified by weapons of mass destruction that did not exist, as a case where a government faced little accountability for false claims; governments and corporations often seem to get away with it. Another student raised Meta, where the consequences have mostly been civil (recent class actions) rather than criminal. The instructor observed that much of what passes for enforcement against disinformation has been civil litigation by private parties, which may end up shaping what platforms do, either to avoid lawsuits or because of the money involved
• Lecture: Disinformation (book 3.5; slides are still a work in progress)
  • What counts. Disinformation is nonaccidentally misleading information (Fallis). Three conditions: it must carry information; it must create false beliefs; and it must do so intentionally. Misstating a fact you simply got wrong (the instructor’s example: saying the Civil War started in 1962) is not disinformation; the term of art for accidentally wrong information is misinformation. Satire is also excluded. One-sided presentation is not disinformation either: a politician who backs a claim with a factoid stripped of context is not strictly propagating false information, though this is a gray area, and many of the sites in the hands-on are political. A student offered an example of something intended to create false beliefs that carries almost no information, which the instructor called a better example than their own; the example itself was not captured
  • Is it illegal? In a few narrow contexts, yes: defamation (false statements that damage a person) and securities fraud (false statements that move a stock price; the instructor recalled Elon Musk getting into trouble with regulators over posts about stocks). Political speech is protected in the United States even when false; the instructor pointed to United States v. Alvarez as the case to read, with the caveat that they are not a lawyer and would check further. In Europe the Digital Services Act regulates the platforms and their systems, not the content of what is said
  • The consequence of leaving it to platforms. If disinformation is not illegal in general, platforms end up deciding what is allowed. Their policies are not consistent with each other (a hands-on next week compares the content-moderation policies of different platforms), and even if they were, enforcing them consistently is of unclear feasibility, and auditing a platform for consistent enforcement is very hard (a research problem the instructor has thought about for a decade). Perceived inconsistency in policy or enforcement produces distrust: the sense that Facebook leans one way, X another, and that a chatbot like Claude has a political bent of its own. It is the same issue as Tuesday’s demo, where two people got different answers and nobody could say whether the difference was benign personalization or something more insidious
  • Pizzagate. Nearly everyone had heard of it, unlike last year’s class. A student told the story: stories spun out of the 2016 Clinton campaign email leaks claimed that Hillary Clinton and the Democratic Party were linked to a child-trafficking ring run from a pizza parlor in Washington, DC; a man showed up at the restaurant to confront the supposed ring and was apprehended before anyone was hurt. Much of the content was produced by people in Macedonia who, by their own account, had no interest in the outcome of the 2016 election; they had discovered that lurid stories about a presidential candidate drew clicks and ad revenue, and they made a lot of money
  • Effects (from Benkler, Faris, and Roberts, Network Propaganda, 2018, which the instructor said has aged well). Three effects of propaganda:
    • Induced misperception: people believe false things. This is the first-order effect everyone thinks about
    • Disorientation: people stop knowing what to believe and distrust sources, including sources they used to consider reputable; cynicism, the inability to tell truth from non-truth. This is the more insidious second-order effect: even if only a small group believes a particular false story, a much larger group spends brain cycles wondering whether to trust anything, including the platform itself (“I heard Facebook is a right-wing thing; should I believe what I read there?”)
    • Distraction: change the subject. This follows directly from flooding; the instructor cited the current president as a master of changing the subject when pinned down
  • Scale is what is new. Orientations toward outlets (the Times as left-leaning, Breitbart as right-leaning, and so on) existed long before social media and could already disorient a reader. What changed is that the number of sources used to be countable on your hands and feet, and AI has made flooding so much cheaper that disorientation and distraction are far more wide-ranging. A further twist, raised by a student that day and suggested as a project topic: when you ask a chatbot a question, it reads a handful of websites and hands you a synthesis without necessarily telling you which sites, so you can no longer consider the source yourself. The instructor asked whether there is sociological work measuring these second-order effects and said they have not followed that literature closely
  • Vocabulary. Sock puppets: fake accounts (bots) that create or amplify content. Astroturfing: a fake grassroots movement, manufactured consensus. The word comes from AstroTurf, the artificial grass invented for the Houston Astrodome after real grass died under the dome’s roof panels. Bogus reviews. Search manipulation and polluting a user’s profile, which changes what that user is shown in the future; a cross-site request forgery attack can do exactly that
  • A taxonomy of manipulation. The instructor’s own grid, built more than ten years ago and motivated by Cialdini’s Influence: The Psychology of Persuasion (a 1970s psychology book turned pop business classic about how marketers and persuaders steer choices). Three levers a manipulator can pull, the choice set (an idea taken from Cialdini), the presentation, and the content, crossed with three stages of the information supply chain: production (newsrooms), dissemination (social networks), and discovery (search)
    • Choice set at production: what a newsroom chooses not to report. The instructor looks at the front pages of the Wall Street Journal and the New York Times daily and they are strikingly different; decisions are being made about what gets covered, presumably driven by clicks and money. A social network removing content and a search engine removing results are the more obvious cases (search comes in a later class)
    • Presentation: the same story is at the top of the Journal and three scrolls down in the Times. Layout and ordering are editorial decisions too
    • Content: a student noted that two outlets can frame the same set of facts in completely different ways, and that what one reader sees differs from what another sees. The instructor asked for the links and said the table is overdue for a refresh
  • Detection (disinformation as the new spam: the SNARE precedent; domain, certificate, and hosting tells that exist before any content is published) was not presented; the instructor said the class had read about it in the book and the slides are there to read
• Hands-on: fake-news-site infrastructure (activity, Part 1)
  • Students worked in pairs for about fourteen minutes on a list of labelled disinformation domains from the infrastructure-detection paper, posted in Slack. The list is five or six years old, so many of the sites now return 404s, but plenty are still live; pairs were told to divide the list between them
  • Questions to consider: what do these sites look like, and do they look like real news sites? Some make an effort; some do not. The older, human-readership framing of the activity asked whether you would actually go to such a site. One answer from an earlier discussion is that readers rarely see the landing page: a friend posts a direct link on Facebook, the reader lands deep in the site, and that is good enough to fool someone
  • The newer framing: if the future of the web is agents reading pages and handing people a synthesis, a site no longer needs to look good to a human at all. It only needs to exist on the web for ingestion by an agent, so the question becomes how to think about sites built to be eaten by bots rather than read by people
  • The instructor said the activity will be modernized for next year: the dead sites in the dataset will be refreshed, and discussion questions about sites that exist to be ingested by agents and LLMs rather than read by humans will be added, since this is only getting more relevant
  • Only the end of the report-back was recorded. One pair described a site that is almost entirely cat content, with a political article slipped in now and then: a site built for engagement (clickbait and algorithmic reach) that carries the occasional payload. The instructor noted this as a tactic they had heard of but not studied. Another pair reported a site whose content had turned into bizarre filler, a reminder that many of these domains have changed hands
  • Next meeting is content moderation, with the third student presentation and breakouts
• Not covered: the four behavioral signatures of a flooding campaign; the “who spreads it and why” slide beyond the Macedonian profit-seekers; the detection slides (SNARE, infrastructure tells); coordinated inauthentic behavior; the AI-generated flooding and Digital Services Act slides beyond the one-line mention above. Breakout A on Community Notes versus fact-checking was not run
• Next: content moderation (book 3.6), with the third student presentation and the LLM sentiment-analysis hands-on; a comparison of platforms’ content-moderation policies is planned for the same week

Meeting 9 (Tue Oct 27)

The recording begins at about 3:45 pm, at the end of housekeeping, and runs to the end of class. The instructor’s opening and the hands-on instructions are captured clearly. The student presentation was recorded from across the room, so much of it is garbled; the notes below keep only what could be reconstructed with confidence. The small-group discussions and most of the hands-on report-back were not captured.

• Housekeeping (only the end was recorded)
  • Project proposals are in students’ private course repositories. Every private repository now has an issue titled “Gradebook” showing which weekly responses and which proposal the staff have received; if something you submitted is missing there, tell the staff
  • Next week has two student presentations: denial of service and disinformation
  • The plan for the day was posted in Slack: reading-response themes, the student presentation on content moderation, a short lecture on how moderation is actually done, and a hands-on using LLMs as moderators
• Reading-response themes (week-5 responses on book 3.6)
  • It is hard to tell that you have been moderated. Removals can be documented and blocking is observable, but a shadowban is difficult to prove. Appeals about visibility are hard because the platform can say the content simply did not get engagement. An open question: how can researchers check moderation for fairness when platforms do not reveal how their systems work?
  • AI moderators can be biased and inconsistent. Human moderators are often assumed to be biased or inconsistent, but automated moderation can be inconsistent too, a point many responses raised
  • Community Notes and community moderation. Notes usually arrive after content has already gone viral, so their effectiveness is limited by speed. The note writers themselves could be biased or could coordinate an attack, and it is not obvious what incentives they have to contribute or to be unbiased
  • Fingerprinting at scale. Consortia such as the Global Internet Forum to Counter Terrorism (GIFCT) maintain shared databases of images and other content used to fingerprint terrorist material. The companies assembling those databases have considerable power over what counts as terrorist or acceptable content, and possibly too much power over what stays online. The class will return to this in a later lecture
  • Copyright and fair use. Automated takedown systems have a poor read on fair use, which raises the question of how fair use survives automated enforcement
  • Who sets the rules about what is acceptable cuts across all of these and was the theme of the day
  • The cat-and-mouse game, automated on both sides. AI is increasingly used for automated moderation, and people posting content can use generative AI to evade it: AI cats and AI mice, with generated content designed to slip past automated moderation on one side and automated moderation systems on the other
• Student presentation: content moderation (third presentation of the term)
  • Platform incentives. Moderation trades off against revenue: provocative content and misinformation can draw engagement, while advertisers may want content removed. The presenters also pointed the class to a shadowban test for X accounts
  • Community rules go beyond safety. The presenters used a subreddit for a television show, which has fourteen rules. The rules fall into categories: safety (no harassment or hate); organization (required post titles and post types, so people can find content they care about); content quality (no self-promotion); spam prevention (a minimum account age or karma to post, which governs who can post, not just what); and user experience, such as no spoilers until the episode has aired on the West Coast, a rule about content that is not harmful at all and that people actually want. Takeaway question: when do community rules become censorship? The same practice of setting community rules can have very different implications depending on context
  • Moderation without removal: demotion. Besides removing content that violates a policy, platforms demote it in algorithmic rankings. The presenters cited research on demoting content judged misleading, and on platforms intervening based on the engagement around a post and the likely consequences of its circulation even when the post is not harmful in itself. Meta states openly that, in addition to removing content, it reduces the distribution of some content, such as misinformation
  • TikTok’s suppression of vulnerable users. Leaked internal moderation guidelines told TikTok moderators to limit the reach of videos from users deemed highly vulnerable to cyberbullying, with examples including facial disfigurement, autism, and Down syndrome: the platform protected those users from bullying by reducing their visibility
  • Activity: who did the moderating? The class guessed which organization was behind three moderation decisions
    • Kiwi Farms, a forum associated with doxxing and real-world harassment campaigns: in September 2022, Cloudflare stopped providing its security and infrastructure services to the site, citing an imminent threat to human life. This is infrastructure-level moderation, very different from a platform deciding what content is allowed
    • youtube-dl, a popular open-source tool for downloading YouTube videos: in October 2020, GitHub took down its repository after a request from the Recording Industry Association of America (RIAA), the trade group representing the major record labels, under the Digital Millennium Copyright Act (DMCA). Few in the class guessed the RIAA. The presenters noted the tool’s legitimate uses: journalists and activists archiving videos, creators saving backup copies of their own uploads, and people on slow or unreliable connections downloading to watch later. Their analogy: youtube-dl is closer to a video cassette recorder than to a piracy tool. This is copyright-based content moderation
    • Pornhub: after reports that the site hosted child sexual abuse material and videos of people who had not consented, Visa and Mastercard stopped processing payments for it. Free content remained available, so the site was not blocked, but creators and subscription services on it could no longer be paid. Neither the payment networks nor Cloudflare in the earlier example had to remove any individual piece of content to exercise this kind of control: moderation through finance
  • Government regulation in the United States. The First Amendment (“Congress shall make no law… abridging the freedom of speech”) protects speech from government restriction, but private platforms keep broad discretion over what they host
    • The Communications Decency Act of 1996 tried to criminalize indecent and patently offensive online communications accessible to minors; the Supreme Court struck those provisions down on First Amendment grounds in 1997. Section 230 survived: “No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.” In practice, a platform is not legally responsible for what its users post
    • Congress tried several more times; the presenters said it was not until 2003 that such a law survived Supreme Court review, and suggested that more favorable public attitudes toward government after 9/11 played a part
    • Regulation is fragmented across states. Florida and Texas passed laws barring platforms from moderating based on a user’s political viewpoint; those laws were in turn challenged on First Amendment grounds and are still being litigated. The irony the presenters drew: courts have struck down both laws that force platforms to moderate and laws that force them not to, on the principle that platforms have editorial rights over what appears on them
    • Lawmaking moves far more slowly than technology
  • Other countries. The United States is less strict than most other countries
    • The European Union’s Digital Services Act requires large platforms to conduct risk assessments, provide transparent mechanisms for removing content, and take down illegal content quickly. The presenters judged it largely effective at promoting transparency and accountability, but its compliance burden falls hardest on smaller platforms with less capital
    • Germany’s network enforcement law (NetzDG) requires platforms to remove manifestly unlawful content within 24 hours of a report. Critics argue it pushes profit-maximizing platforms to over-remove rather than risk penalties; it relies on user reporting; and it does not address algorithmic amplification or personalization
    • In authoritarian regimes, censorship is direct and some platforms are run by the state, which raises the question of whether that is content moderation at all. In many developing countries, regulation is inconsistent, often copied from other countries, and weakly enforced
    • Australia is the first country to ban children under 16 from social media, penalizing the platforms rather than the users. Enforcement is weak because many sites do not check age, and age checks that require identification raise privacy concerns
    • The question this leaves: how should platforms that operate internationally comply with so many different regimes?
  • Discussion. The presenters asked what characterizes content moderation, who should be allowed to do it and for what reasons, and where it ends. A student argued that a platform that chooses not to moderate on political grounds is still moderating, since it is still choosing what to leave up; a presenter agreed. Another student said the GitHub takedown of youtube-dl looked like plain censorship. The rest of the discussion was not captured
• Lecture: how moderation is done was not presented. The instructor said the presentation and discussion left little for the slides to add, and the slides (fingerprinting versus prediction, why prediction is brittle, auditing a system you cannot see) are posted to read
• Hands-on: LLMs as content moderators (activity, Part 1; about ten minutes)
  • The instructor used to run this activity with Google’s Perspective API, which did a poor job: you could trick it by changing the language, the context, or a few keywords. It now uses large language models, which seem to be getting very good at this; in a few years the activity may not be worth running at all. Students could use the university’s PhoenixAI tool
  • In pairs or small groups: give a model a prompt such as “please evaluate the following content for toxicity, hate speech, threats, or harassment” (or swap in copyright or disinformation), add test text you write yourself or generate with another model, and compare. Things to try: borderline cases, sarcasm, satire, other languages, full sentences versus phrases, and prefacing the same text with different context, which has been known to change the answer. Nobody had to type hate speech; there are plenty of other categories to test
  • Report-back, partly captured: one group tested a sentence whose meaning flips with a comma, a long-standing favorite example. Another group got an “allowed” verdict from one chatbot, along with its rationale for why the text was not threatening, while Claude said the same text should be blocked. The instructor asked them to run the text through a fresh Claude session to see whether a second session of the same model would agree. The result was not captured before class ended
  • Platform moderation policies and Breakout B were not run
• Next: personalization (book 3.7), Thursday October 29