NVIDIA GPU Optimisation Tools

RidgeRun CUDA Optimisation Guide
GPU Architecture
Execution process Memory hierarchy Communication & Concurrency
Optimisation Workflow
Tools CUDA Memcheck CUDA Profiler Computational Budget Tool
Optimisation Recipes
Types of optimisations Coarse optimisations Workload offloading Problem size Communication overlapping Correct memory access patterns Inter-thread communication Memory management Fine optimisations Increase arithmetic intensity Function approximation Condition and loops replacement Inlining
Common pitfalls when optimising
Examples
Empirical Experiments
Simple bounding test Multi-threaded bounding test
Contact Us

Continuing with the workflow, profiling, and debugging tools are crucial for optimisation. Having basic knowledge is critical for finding optimisation opportunities and debugging issues that can arise.

CUDA debugger and checker

Most of the issues can be identified by using execution tracing (a.k.a placing printf everywhere). However, that's suboptimal in some cases and, for debugging optimisation issues, it is better to know where they are and how to fix them without working around them. Additionally, it is a good practice to identify the issues that the original application has before any attempt. It will help you to (1) fix them, and (2) avoid headaches when introducing the optimisations.

Additionally, without implementing any change, it is necessary to profile the application to identify the current hotspots and bottlenecks. There are three possible paths to follow when optimising: (1) the GPU computation, (2) the memory access, and (3) the I/O. Depending on your profiling results, you should check them.