CUDA's non-determinism is in scheduling and memory access, not computation itself. This is the same guarantees as you get with a CPU; the computation is deterministic, but the point at which you work ...
GPUs do provide the same IEEE 754 guarantees on determinism when it comes to floats as CPUs do; they're fully deterministic machines in that regard. The reason people talk about GPUs as ...