The ability to build progressively on the achievements of earlier generations is central to human uniqueness, but experimental investigations of this cumulative cultural evolution lack real-world complexity. Here, we studied the dynamics of cumulative culture using a large-scale data set from online collaborative programming competitions run over 14 years. We show that, within each contest population, performance increases over time through frequent ‘tweaks’ of the current best entry and rare innovative ‘leaps’ (successful tweak:leap ratio = 16:1), the latter associated with substantially greater variance in performance. Cumulative cultural evolution reduces technological diversity over time, as populations focus on refining high-performance solutions. While individual entries borrow from few sources, iterative copying allows populations to integrate ideas from many sources, demonstrating a new form of collective intelligence. Our results imply that maximising technological progress requires accepting high levels of failure.