Don\u2019t worry if you\u2019re not up to speed on the ins and outs of ML, since we won\u2019t be using it in this article. Instead, I\u2019ll show you how to use the PyTorch library to access and use the capabilities of your GPU. We\u2019ll compare the run times of Python<\/a> programs using the popular numerical library NumPy,<\/strong> running on the CPU, with equivalent code using PyTorch on the GPU.\u00a0<\/p>\n Before continuing, let\u2019s quickly recap what a GPU and Pytorch<\/a> are.<\/p>\n A GPU is a specialised electronic chip initially designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. Its utility as a rapid image manipulation device was based on its ability to perform many calculations simultaneously, and it\u2019s still used for that purpose.<\/p>\n However, GPUs have recently become invaluable in machine learning, large language model training and development. Their inherent ability to perform highly parallelizable computations makes them ideal workhorses in these fields, as they employ complex mathematical models and simulations.<\/p>\n PyTorch is an open-source machine learning library developed by Facebook\u2019s AI Research Lab (FAIR). It\u2019s widely used for natural language processing and computer vision applications. Two of the main reasons that Pytorch can be used for GPU operations are,<\/p>\n In summary, PyTorch\u2019s support for GPU operations through CUDA and its efficient tensor manipulation capabilities make it an excellent tool for developing GPU-accelerated Python functions with high computational demands.\u00a0<\/p>\n As we\u2019ll show later on, you don\u2019t have<\/em><\/strong> to use PyTorch to develop machine learning models or train large language models.<\/p>\n In the rest of this article, we\u2019ll set up our development environment, install PyTorch and run through a few examples where we\u2019ll compare some computationally heavy PyTorch implementations with the equivalent numpy implementation and see what, if any, performance differences we find.<\/p>\n You need an Nvidia GPU on your system. To check your GPU, issue the following command at your system prompt. I\u2019m using the Windows Subsystem for Linux (WSL). <\/p>\n If that command isn\u2019t recognised and you\u2019re sure you have a GPU, it probably means you\u2019re missing an NVIDIA driver. Just follow the rest of the instructions in this article, and it should be installed as part of that process.<\/p>\n While PyTorch installation packages can include CUDA libraries, your system must still install the appropriate NVIDIA GPU drivers. These drivers are necessary for your operating system to communicate with the graphics processing unit (GPU) hardware. The CUDA toolkit includes drivers, but if you\u2019re using PyTorch\u2019s bundled CUDA, you only need to ensure that your GPU drivers are current.<\/p>\n Click this link<\/a> to go to the NVIDIA website and install the latest drivers compatible with your system and GPU specifications<\/span>.<\/p>\n As a best practice, we should set up a separate development environment for each project. I use conda, but use whatever method suits you.<\/p>\n If you want to go down the conda route and don\u2019t already have it, you must install Miniconda (recommended) or Anaconda first.\u00a0<\/em><\/p>\n Please note that, at the time of writing, PyTorch currently only officially supports Python versions 3.8 to 3.11.<\/em><\/p>\n<\/blockquote>\n Now activate your new environment.<\/p>\n We now need to get the appropriate conda install command for PyTorch. This will depend on your operating system, chosen programming language, preferred package manager, and CUDA version.\u00a0<\/p>\n Luckily, Pytorch provides a useful web interface that makes this easy to set up. So, to get started, head over to the Pytorch website at\u2026<\/p>\n https:\/\/pytorch.org<\/a><\/p>\n Click on the Click on each box in the appropriate position for your system and specs. As you do, you\u2019ll see that the command in the For me, this was:-<\/p>\n We\u2019ll install Jupyter, Pandas, and Matplotlib to enable us to run our Python code in a notebook with our example code.<\/p>\n Now type in Near the bottom, there will be a URL that you should copy and paste into your browser to initiate the Jupyter Notebook.<\/p>\n Your URL will be different to mine, but it should look something like this:-<\/p>\n The first thing we\u2019ll do is test our setup. Please enter the following into a Jupyter cell and run it.<\/p>\n You should see a similar output to the following.<\/p>\n Additionally, to check if your GPU driver and CUDA are enabled and accessible by PyTorch, run the following commands:<\/p>\n This should output If everything is okay, we can proceed to our examples. If not, go back and check your installation processes.<\/p>\n NB In the timings below, I ran each of the Numpy and PyTorch processes several times in succession and took the best time for each. This does favour the PyTorch runs somewhat as there is a small overhead on the very first invocation of each PyTorch run but, overall, I think it\u2019s a fairer comparison.<\/em><\/strong><\/p>\n<\/blockquote>\n In this example, we set up two large, identical one-dimensional arrays and perform a simple addition to each array element.<\/p>\n We see a slight improvement when using PyTorch over Numpy, but we missed one crucial point. We haven\u2019t used the GPU because our PyTorch tensor data is still in CPU memory.\u00a0<\/p>\n To move the data to the GPU memory, we need to add the After re-running with the changes we get,<\/p>\n That\u2019s more like it, a greater than 10x speed up.\u00a0<\/p>\n For this example, we\u2019ll multiply multi-dimensional matrices using the built-in matmul<\/strong> operations available in the PyTorch and Numpy libraries. Each array will be 10000 x 10000 and contain random floating-point numbers between 1 and 100.<\/p>\n Now for the PyTorch version.<\/p>\n The PyTorch run was 20 times better this time than the NumPy run. Great stuff.<\/p>\n Sometimes, not all of your processing can be done on a GPU. An everyday use case for this is graphing data. Sure, you can manipulate your data using the GPU, but often the next step is to see what your final dataset looks like using a plot.<\/p>\n You can\u2019t plot data if it resides in the GPU memory, so you must move it back to CPU memory before calling your plotting functions. Is it worth the overhead of moving large chunks of data from the GPU to the CPU? Let\u2019s find out.<\/p>\n In this example, we will solve this polar equation for values of \u03b8 between 0 and 2\u03c0 in (x, y) coordinate terms and then plot out the resulting graph.<\/p>\n Don\u2019t get too hung up on the math. It\u2019s just an equation that, when converted to use the x, y coordinate system and solved, looks nice when plotted.<\/p>\n For even a few million values of x and y, Numpy can solve this in milliseconds, so to make it a bit more interesting, we\u2019ll use 100 million (x, y) coordinates.<\/p>\n Here is the numpy code first.<\/p>\n Here is the output. Would you have guessed beforehand that it would look like this? I sure wouldn\u2019t have!<\/p>\n Now, let\u2019s see what the equivalent PyTorch implementation looks like and how much of a speed-up we get.<\/p>\n And our output again.<\/p>\n The calculation part was about 10 times more than the numpy calculation. The data plotting took around the same time using both the PyTorch and NumPy versions, which was expected since the data was still in CPU memory then, and the GPU played no further part in the processing.<\/p>\n But, overall, we shaved about 40% off the total run-time, which is excellent.<\/p>\n This article has demonstrated how to leverage an NVIDIA GPU using PyTorch\u2014a machine learning library typically used for AI applications\u2014to accelerate non-ML numerical Python code. It compares standard NumPy (CPU-based) implementations with GPU-accelerated PyTorch equivalents to show the performance benefits of running tensor-based operations on a GPU.<\/p>\n You don\u2019t need to be doing machine learning to benefit from PyTorch. If you can access an NVIDIA GPU, PyTorch provides a simple and effective way to significantly speed up computationally intensive numerical operations\u2014even in general-purpose Python code.<\/p>\n The post Use PyTorch to Easily Access Your GPU<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\nWhat is a\u00a0GPU?<\/h2>\n
What is\u00a0PyTorch?<\/h2>\n
\n
Pre-requisites<\/h2>\n
\n
$ nvidia-smi\n\n>>\n(base) PS C:Usersthoma> nvidia-smi\nFri Mar 22 11:41:34 2024\n+-----------------------------------------------------------------------------------------+\n| NVIDIA-SMI 551.61 Driver Version: 551.61 CUDA Version: 12.4 |\n|-----------------------------------------+------------------------+----------------------+\n| GPU Name TCC\/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |\n| Fan Temp Perf Pwr:Usage\/Cap | Memory-Usage | GPU-Util Compute M. |\n| | | MIG M. |\n|=========================================+========================+======================|\n| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N\/A |\n| 32% 24C P8 9W \/ 285W | 843MiB \/ 12282MiB | 1% Default |\n| | | N\/A |\n+-----------------------------------------+------------------------+----------------------+\n+-----------------------------------------------------------------------------------------+\n| Processes: |\n| GPU GI CI PID Type Process name GPU Memory |\n| ID ID Usage |\n|=========================================================================================|\n| 0 N\/A N\/A 1268 C+G ...tilityHPSystemEventUtilityHost.exe N\/A |\n| 0 N\/A N\/A 2204 C+G ...ekyb3d8bbwePhoneExperienceHost.exe N\/A |\n| 0 N\/A N\/A 3904 C+G ...calMicrosoftOneDriveOneDrive.exe N\/A |\n| 0 N\/A N\/A 7068 C+G ...CBS_cw5n\netc ..<\/code><\/pre>\n\n
Setting up our development environment<\/h2>\n
\n
#create our test environment\n(base) $ conda create -n pytorch_test python=3.11 -y<\/code><\/pre>\n(base) $ conda activate pytorch_test<\/code><\/pre>\nGet Started <\/code>link near the top of the screen. From there, scroll down a little until you see this,<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/05\/image-115-1024x754.png?resize=1024%2C754&ssl=1\")
Run this Command<\/code> output field changes dynamically. When you\u2019re done making your choices, copy the final command text shown and type it into your command window prompt.\u00a0<\/p>\n(pytorch_test) $ conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia -y<\/code><\/pre>\n(pytroch_test) $ conda install pandas matplotlib jupyter -y<\/code><\/pre>\njupyter notebook <\/code>into your command prompt. You should see a jupyter notebook open in your browser. If that doesn\u2019t happen automatically, you\u2019ll likely see a screenful of information after the jupyter notebook <\/code>command.<\/p>\nhttp:\/\/127.0.0.1:8888\/tree?token=3b9f7bd07b6966b41b68e2350721b2d0b6f388d248cc69da<\/code><\/pre>\nTesting our\u00a0setup<\/h4>\n
import torch\nx = torch.rand(5, 3)\nprint(x)<\/code><\/pre>\ntensor([[0.3715, 0.5503, 0.5783],\n [0.8638, 0.5206, 0.8439],\n [0.4664, 0.0557, 0.6280],\n [0.5704, 0.0322, 0.6053],\n [0.3416, 0.4090, 0.6366]])<\/code><\/pre>\nimport torch\ntorch.cuda.is_available()<\/code><\/pre>\nTrue<\/code> if all is OK.\u00a0<\/p>\n\n
Example 1\u200a\u2014\u200aA simple array math operation.<\/h3>\n
import numpy as np\nimport torch as pt\n\nfrom timeit import default_timer as timer \n \n\n#func1 will run on the CPU \ndef func1(a): \n a+= 1 \n \n#func2 will run on the GPU\ndef func2(a): \n a+= 2\n\nif __name__==\"__main__\": \n n1 = 300000000 \n a1 = np.ones(n1, dtype = np.float64) \n\n # had to make this array much smaller than\n # the others due to slow loop processing on the GPU\n n2 = 300000000 \n a2 = pt.ones(n2,dtype=pt.float64)\n\n start = timer() \n func1(a1) \n print(\"Timing with CPU:numpy\", timer()-start) \n \n start = timer() \n func2(a2) \n #wait for all calcs on the GPU to complete\n pt.cuda.synchronize()\n print(\"Timing with GPU:pytorch\", timer()-start) \n print()\n\n print(\"a1 = \",a1)\n print(\"a2 = \",a2)<\/code><\/pre>\nTiming with CPU:numpy 0.1334826999955112\nTiming with GPU:pytorch 0.10177790001034737\n\na1 = [2. 2. 2. ... 2. 2. 2.]\na2 = tensor([3., 3., 3., ..., 3., 3., 3.], dtype=torch.float64)<\/code><\/pre>\ndevice='cuda'<\/code> directive when creating the tensor. Let\u2019s do that and see if it makes a difference.<\/p>\n# Same code as above except \n# to get the array data onto the GPU memory\n# we changed\n\na2 = pt.ones(n2,dtype=pt.float64)\n\n# to\n\na2 = pt.ones(n2,dtype=pt.float64,device='cuda')<\/code><\/pre>\nTiming with CPU:numpy 0.12852740001108032\nTiming with GPU:pytorch 0.011292399998637848\n\na1 = [2. 2. 2. ... 2. 2. 2.]\na2 = tensor([3., 3., 3., ..., 3., 3., 3.], device='cuda:0', dtype=torch.float64)<\/code><\/pre>\nExample 2\u2014A slightly more complex array operation.<\/h3>\n
# NUMPY first\nimport numpy as np\nfrom timeit import default_timer as timer\n\n# Set the seed for reproducibility\nnp.random.seed(0)\n# Generate two 10000x10000 arrays of random floating point numbers between 1 and 100\nA = np.random.uniform(low=1.0, high=100.0, size=(10000, 10000)).astype(np.float32)\nB = np.random.uniform(low=1.0, high=100.0, size=(10000, 10000)).astype(np.float32)\n# Perform matrix multiplication\nstart = timer() \nC = np.matmul(A, B)\n\n# Due to the large size of the matrices, it's not practical to print them entirely.\n# Instead, we print a small portion to verify.\nprint(\"A small portion of the result matrix:n\", C[:5, :5])\nprint(\"Without GPU:\", timer()-start)<\/code><\/pre>\nA small portion of the result matrix:\n [[25461280. 25168352. 25212526. 25303304. 25277884.]\n [25114760. 25197558. 25340074. 25341850. 25373122.]\n [25381820. 25326522. 25438612. 25596932. 25538602.]\n [25317282. 25223540. 25272242. 25551428. 25467986.]\n [25327290. 25527838. 25499606. 25657218. 25527856.]]\n\nWithout GPU: 1.4450852000009036<\/code><\/pre>\nimport torch\nfrom timeit import default_timer as timer\n\n# Set the seed for reproducibility\ntorch.manual_seed(0)\n\n# Use the GPU\ndevice = 'cuda'\n\n# Generate two 10000x10000 tensors of random floating point \n# numbers between 1 and 100 and move them to the GPU\n#\nA = torch.FloatTensor(10000, 10000).uniform_(1, 100).to(device)\nB = torch.FloatTensor(10000, 10000).uniform_(1, 100).to(device)\n\n# Perform matrix multiplication\nstart = timer()\nC = torch.matmul(A, B)\n\n# Wait for all current GPU operations to complete (synchronize)\ntorch.cuda.synchronize() \n\n# Due to the large size of the matrices, it's not practical to print them entirely.\n# Instead, we print a small portion to verify.\nprint(\"A small portion of the result matrix:n\", C[:5, :5])\nprint(\"With GPU:\", timer() - start)<\/code><\/pre>\nA small portion of the result matrix:\n [[25145748. 25495480. 25376196. 25446946. 25646938.]\n [25357524. 25678558. 25675806. 25459324. 25619908.]\n [25533988. 25632858. 25657696. 25616978. 25901294.]\n [25159630. 25230138. 25450480. 25221246. 25589418.]\n [24800246. 25145700. 25103040. 25012414. 25465890.]]\n\nWith GPU: 0.07081239999388345<\/code><\/pre>\nExample 3\u200a\u2014\u200aCombining CPU and GPU\u00a0code.<\/h3>\n
![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/05\/image-116.png?ssl=1\")
<\/figure>\n%%time\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom time import time as timer\n\nstart = timer()\n\n# create an array of 100M thetas between 0 and 2pi\ntheta = np.linspace(0, 2*np.pi, 100000000)\n\n# our original polar formula\nr = 1 + 3\/4 * np.sin(3*theta)\n\n# calculate the equivalent x and y's coordinates \n# for each theta\nx = r * np.cos(theta)\ny = r * np.sin(theta)\n\n# see how long the calc part took\nprint(\"Finished with calcs \", timer()-start)\n\n# Now plot out the data\nstart = timer()\nplt.plot(x,y)\n\n# see how long the plotting part took\nprint(\"Finished with plot \", timer()-start)<\/code><\/pre>\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/05\/image-117-1024x698.png?resize=1024%2C698&ssl=1\")
<\/figure>\n%%time\nimport torch as pt\nimport matplotlib.pyplot as plt\nfrom time import time as timer\n\n# Make sure PyTorch is using the GPU\ndevice = 'cuda'\n\n# Start the timer\nstart = timer()\n\n# Creating the theta tensor on the GPU\ntheta = pt.linspace(0, 2 * pt.pi, 100000000, device=device)\n\n# Calculating r, x, and y using PyTorch operations on the GPU\nr = 1 + 3\/4 * pt.sin(3 * theta)\nx = r * pt.cos(theta)\ny = r * pt.sin(theta)\n\n# Moving the result back to CPU for plotting\nx_cpu = x.cpu().numpy()\ny_cpu = y.cpu().numpy()\n\npt.cuda.synchronize()\nprint(\"Finished with calcs\", timer() - start)\n\n# Plotting\nstart = timer()\nplt.plot(x_cpu, y_cpu)\nplt.show()\n\nprint(\"Finished with plot\", timer() - start)<\/code><\/pre>\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/05\/image-118-1024x728.png?resize=1024%2C728&ssl=1\")
<\/figure>\nSummary<\/h2>\n