Getting There, and Proving It

In October 2014 I wrote a blog post (opens in a new tab) after an online round of ICPC, the international college programming contest. One of its problems had been used in an earlier contest, and the code was online, so many of the solutions on the scoreboard weren’t the teams’ own work. I called it “heart-breaking”. A rank is meant to be a receipt for the work behind it. Some of those weren’t real.

This month the same contest is checking its scoreboard again. The ICPC India online round ran on 3 October. Its final list was due on 20 October, after seventeen days of organisers working out which results they could trust, and that day they moved it to 30 October, citing “the large number of suspicious submissions”. For my generation, an honest rank was proof of work. In the AI era a rank alone doesn’t stand. This year’s round shows why, and what a student, a teacher or an engineer can show instead.

I knew what was behind my own rank. Before my team reached the World Finals in 2015, I spent months of 2013 on Topcoder, a site that runs short online contests, stuck just below the rating that would let me into its harder division. At the Finals in Marrakech, there was no question whose code it was: one room, no internet, three of us at one keyboard in plain view. Our result there was a receipt for a lot of evenings of being stuck.

In 2014, copying needed someone else’s old solution to exist. Now, for many contest problems, an AI can write a new one in seconds. From the stats reported this year, my estimate is that 30 to 40% of accepted solutions came this way: problems we’d expect a few dozen teams to solve had hundreds of accepted answers. A correct answer on a scoreboard no longer tells you who did the work. So the organisers checked history first. Teams that solved four or more problems had to send links to their records on public practice sites, or drop off the provisional list.

It hurts. I think most about the students who sat this round honestly. One team posted on Codeforces that six clean solves, worth about rank 100 last year by their estimate, looked set to land between 400 and 500. Out of nearly 6,000 teams, that can be the difference between qualifying for a regional site and not. Some honest teams dropped off the provisional list because they hadn’t sent practice records in time. They did the work, and the scoreboard still couldn’t tell them apart from the teams that didn’t. I still feel dejected about it, and I worry about what happens if the system doesn’t penalise the ones who cheated.

So we need more receipts than a rank.

The one I trust most is consistency: doing it contest after contest, honestly. A history can be produced too, but it is much harder to fake than one result. You might cheat once, twice, or for a stretch, but it is hard to replace years of human effort, and true effort shows. Demanding that trail has a cost too, as the honest teams who missed the deadline found out. History can earn someone the benefit of the doubt, but it shouldn’t be the only gate. The other receipt is being able to explain your work and defend it when someone cross-questions you. The organisers kept that check in reserve: a team under question could be asked to solve similar problems under supervision. It brings back the room in Marrakech.

As a teacher, I see the same scoreboard in classrooms now.

Marking finished work isn’t enough when the finished work may not be the student’s. We have to set assessments knowing what an AI can easily return, and ask what a student can reproduce in a proctored room with no internet. Then cross-question it. You can’t question every answer, but you can question the ones that surprise you. And questioning does more than catch borrowed code; it shows you how a student actually thinks. When a student’s code fails on a backtracking problem, where a program has to try options and undo the wrong ones, the obvious guess is that they don’t understand the method. Ask them to walk through it line by line, and sometimes the real gap is underneath: they hadn’t formed a clear picture of the call stack, the way a program keeps track of which step it’s on. The finished code wouldn’t have shown that. The questions did.

It is the same for software engineers.

You can submit what an AI produced, but if you didn’t learn from it, it doesn’t matter for long. Your receipts there are the git log, the record of every change to a piece of code, and whether you can explain each one when asked.

Three days before this year’s round, Vipin Pavithran, who guided me through my ICPC years, wrote a letter to contestants (opens in a new tab). Its one boxed request was “Don’t cheat yourself.” My advisor during World Finals prep was humblefool, the first Indian to reach Topcoder’s red rating band, and his advice was to chase excellence, not accolades. The receipt is in the process and in the skill that stays after the contest, not in the few hours of rank an AI’s answer can buy.

Sources

Chapters