NVIDIA GPU Optimisation Workflow

RidgeRun CUDA Optimisation Guide
GPU Architecture
Execution process Memory hierarchy Communication & Concurrency
Optimisation Workflow
Tools CUDA Memcheck CUDA Profiler Computational Budget Tool
Optimisation Recipes
Types of optimisations Coarse optimisations Workload offloading Problem size Communication overlapping Correct memory access patterns Inter-thread communication Memory management Fine optimisations Increase arithmetic intensity Function approximation Condition and loops replacement Inlining
Common pitfalls when optimising
Examples
Empirical Experiments
Simple bounding test Multi-threaded bounding test
Contact Us

There are several optimisation ways and paths. Perhaps, the most intuitive one involves, in relevance order, the following steps:

1. Identify the problems and hotspots of the application
2. Identify the optimisation opportunities: focus on the coarse optimisations
3. Looping on the coarse optimisations: stop until reaching the goal
   1. Apply the optimisation
   2. Reprofile the application for more coarse optimisations
4. Identify more optimisation opportunities: focus on the fine optimisations
   1. Evaluate the relevance of the optimisation and possible impact. More aggressive will be always preferred
   2. Apply the optimisation
   3. Reprofile the application

The process can be graphically seen as follows:

A really basic synthesis of an optimisation flow for CUDA

In the next section, the common optimisations and their classification are presented.