NVIDIA - GPU - Control Commands - How to use nvidia-smi effectively. - Must read.

Dear Researchers/NVIDIA-Graphics Card users,
In this post I am going to share a command which will allow you to use NVIDIA-graphics card more efficiently. Many times we are faced with getting information such as who is using the NVIDIA-graphics card, how much memory is being used for a graphics card and whom to tell "Please kill your jobs I have something important coming up!!!"
The following command took me about 1.5 hours to create since there was no information on the internet on a command like this.
The command is long - So save it in your bashrc file with your favorite alias.
nvidia-smi && nvidia-smi pmon -c 1 | awk -F ' ' '{print $2}' | xargs -0 > t1 && export i=0; export l2="-"; while IFS='' read -r line;do if [[ "$i" -lt 2 || -z "$line" || "$line" == $l2 ]];then echo $i; else ps -p "$line" -o pid,vsz=MEMORY -o user,group=GROUP -o comm,args=ARGS | awk '{for ( x=1 ; x<=1 ; x++ ) { printf("%s\t",$x) } for (x=2;x<=2;x++) { if(NR>1) { printf("%13.2fMb\t",hr=$x/1024) } else { printf("\t%s\t",$x) } } for ( x=3 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }'; fi; i=$((i+1)) ;done < t1 && rm t1
Output will look something like this
Normal NVIDIA-SMI command and then for each individual command information as follows
PID MEMORY USER GROUP COMMAND ARGS

No comments:

Post a Comment