How to check the health of a GPU

I have a GTX 1080Ti which I have been using as my personal deep learning card for about 3,5 years. Lately, I'm getting a lot of obscure CUDA errors. Training and inference code that was running fine before, is now throwing errors after running for a few minutes (or my pc shuts down).

It's usually a version of:

CUDA: unspecified launch failure 

I'm starting to suspect that it's a hardware problem with my GPU. I was wondering if there are ways to confirm this? (NOTE: apart from these errors the GPU seems fine - no visual artifacts on my monitor or anything like that.)

For example I have tried:

  • (test didn't show any fault with my GPU)

Are there any other ways?

2

1 Answer

Training and inference code that was running fine before, is now throwing errors after running for a few minutes

Sounds like the card could be failing.


(or my pc shuts down)

Sounds like the power supply could be failing


I would start by replacing the cheaper option, the power supply.

It's possible that a faulty power supply is causing GPU faults.

Your Answer

Sign up or log in

Sign up using Google Sign up using Facebook Sign up using Email and Password

Post as a guest

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

You Might Also Like