Posts

nvidia-smi: Control Your GPUs

Most users know how to check the status of their CPUs, see how much system memory is free, or find out how much disk space is free. In contrast, keeping tabs on the health and status of GPUs has historically been more difficult. If you don’t know where to look, it can even be difficult to determine the type and capabilities of the GPUs in a system. Thankfully, NVIDIA’s latest hardware and software tools have made good improvements in this respect. The tool is NVIDIA’s System Management Interface ( nvidia-smi ). Depending on the generation of your card, various levels of information can be gathered. Additionally, GPU configuration options (such as ECC memory capability) may be enabled and disabled. As an aside, if you find that you’re having trouble getting your NVIDIA GPUs to run GPGPU code, nvidia-smi can be handy. For example, on some systems the proper NVIDIA devices in /dev are not created at boot. Running a simple nvidia-smi query as root will initialize all the c...

Github update - wrongly committed large file.

I added a large file to a git repository (102Mb), commited and push and got an error due to size limit limitations on github remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com. remote: error: Trace: 7d51855d4f834a90c5a5a526e93d2668 remote: error: See http://git.io/iEPt8g for more information. remote: error: File coverage/sensitivity/simulated.bed is 102.00 MB; this exceeds GitHub's file size limit of 100.00 MB 1 2 3 4 remote : error : GH001 : Large files detected . You may want to try Git Large File Storage - https : / / git - lfs .github .com . remote : error : Trace : 7d51855d4f834a90c5a5a526e93d2668 remote : error : See http : / / git .io / iEPt8g for more information . remote : error : File coverage / sensitivity / simulated .bed is 102.00 MB ; this exceeds GitHub ' s file size limit of 100.00 MB Her...

How to download very large file from Google Drive?

Here are step-by-step instructions to download a file from Google Drive using the command line API if the file is shared privately and needs authentication. Get the file ID: Go to your Google Drive in your browser. Right-click (or control-click) the file you want to download and click “Get shareable link”. The link looks like this: https : //drive.google.com/open?id=XXXXX . Make note of the file ID “XXXXX”; you will be needing it below. Get an OAuth token: Go to OAuth 2.0 Playground In the “Select the Scope” box, scroll down, expand “Drive API v3”, and select https : //www.googleapis.com/auth/drive.readonly Click “Authorize APIs” and then “Exchange authorization code for tokens”. Copy the “Access token”; you will be needing it below. Download the file from the command line: If using OS X or Linux, open the “Terminal” program and enter the following command. curl - H "Authorization: Bearer YYYYY"   https : //www.googleapis.com/drive/v3/files/XXXXX?a...

Career Advice

Career advice:‬ ‪1 you are not your job‬ ‪2 I can > IQ‬ ‪3 do not speak poorly about others‬ ‪4 there is no career ladder‬; lift others and you become the ladder ‪5 know when it is time to leave‬ ‪6 you do not need a title to lead‬ ‪7 always meet deadlines‬ ‪8 deliver outside of your job description‬ ‪9 stay teachable‬ ‪10 share the credit‬

Leetcode Strobogrammatic Number

[LeetCode] Strobogrammatic Number A strobogrammatic number is a number that looks the same when rotated 180 degrees (looked at upside down). Write a function to determine if a number is strobogrammatic. The number is represented as a string. Example 1: Input: "69" Output: true Example 2: Input: "88" Output: true Example 3: Input: "962" Output: false This question defines a symmetrical number, that is, a number is rotated 180 degrees as the original, that is, it is the same as the reverse, such as 609, the reverse is still 609, etc. There are actually few numbers that satisfy this condition, only 0, 1, 8, 6, 9. This question can actually be regarded as a special case of seeking the number of replies. We still use double pointers to detect, so if the first and last two numbers are equal, then only one of them is 0,1,8, if If they are not equal, one must be 6 and one is 9, or one is 9 and one is 6, and all other cases...

Get a job at Google

Original post -  https://steve-yegge.blogspot.com/2008/03/get-that-job-at-google.html Get that job at Google I've been meaning to write up some tips on interviewing at Google for a good long time now. I keep putting it off, though, because it's going to make you mad. Probably. For some statistical definition of "you", it's very likely to upset you. Why? Because... well, here, I wrote a little ditty about it: Hey man, I don't know that stuff Stevey's talking aboooooout If my boss thinks it's important I'm gonna get fiiiiiiiiiired Oooh yeah baaaby baaaay-beeeeee.... I didn't realize this was such a typical reaction back when I first started writing about interviewing, way back at other companies. Boy-o-howdy did I find out in a hurry. See, it goes like this: Me: blah blah blah, I like asking question X in interviews, blah blah blah... You: Question X? Oh man, I haven't heard about X since college! I've never needed it for my jo...

Writing Google AI residency letter

Writing a Google AI Residency Cover Letter Guest post by Katherine Lee* and Ben Eysenbach* *Equal contribution 2019 is an exciting year to start a career in machine learning (ML) and artificial intelligence (AI)[ 1 ]. Whether coming straight out of school or switching from a job in another field, recent advances in ML have created opportunities to apply ML to real-world problems, design and analyze algorithms for ML, and study the interpretability and fairness of our algorithms. While the traditional path to ML research is through a PhD in ML, Google AI and many other ML research labs have introduced “AI Residency” programs as an alternative path. Similar to ML PhD students, Residents learn how to pursue a research project, what questions to ask, and how to use TensorFlow and other ML tools. While there are many good resources on how to apply to PhD programs, few references discuss applying to AI Residency programs. In applying to these programs, you'll need to c...

NVIDIA graphic card and CUDA architecture pairs

Matching SM architectures (CUDA arch and CUDA gencode) for various NVIDIA cards I’ve seen some confusion regarding NVIDIA’s nvcc sm flags and what they’re used for: When compiling with NVCC, the arch flag (‘ -arch ‘) specifies the name of the NVIDIA GPU architecture that the CUDA files will be compiled for. Gencodes (‘ -gencode ‘) allows for more PTX generations, and can be repeated many times for different architectures. When should different ‘gencodes’ or ‘cuda arch’ be used? When you compile CUDA code, you should always compile only one ‘ -arch ‘ flag that matches your most used GPU cards. This will enable faster runtime, because code generation will occur during compilation. If you only mention ‘ -gencode ‘, but omit the ‘ -arch ‘ flag, the GPU code generation will occur on the JIT compiler by the CUDA driver. When you want to speed up CUDA compilation, you want to reduce the amount of irrelevant ‘ -gencode ‘ flags. However, sometimes you may wish to have better CUDA back...