I have a GTX 1080Ti which I have been using as my personal deep learning card for about 3,5 years. Lately, I'm getting a lot of obscure CUDA errors. Training and inference code that was running fine before, is now throwing errors after running for a few minutes (or my pc shuts down).
It's usually a version of:
CUDA: unspecified launch failure I'm starting to suspect that it's a hardware problem with my GPU. I was wondering if there are ways to confirm this? (NOTE: apart from these errors the GPU seems fine - no visual artifacts on my monitor or anything like that.)
For example I have tried:
Are there any other ways?
21 Answer
Training and inference code that was running fine before, is now throwing errors after running for a few minutes
Sounds like the card could be failing.
(or my pc shuts down)
Sounds like the power supply could be failing
I would start by replacing the cheaper option, the power supply.
It's possible that a faulty power supply is causing GPU faults.