Instant Loss on a Budget