Building nVidia's cuspatial

See rapidsai/cuspatial bug report #1493

Working version (10 Dec 2024)

Ensure CUDA 12.6 symbolic links are ok. Eg include should point to targets/x86_64-linux/include

/opt/cuda/cuda-12.6/include -> targets/x86_64-linux/include

10^th December cuspatial now compiles, links, builds and run gtests ok.
Compile, link and install build using

setenv CUSPATIAL_HOME `pwd`/cuspatial
setenv PATH /opt/cuda/cuda-12.6/bin/:"$PATH"
setenv LD_LIBRARY_PATH /opt/cuda/cuda-12.6/lib64:"$LD_LIBRARY_PATH"
echo "Since not restricting the number of threads crashed the machine, do specify -j1"
echo "One thread only, this will be slow (3+ hours)"
setenv PARALLEL_LEVEL "-j1"
cd $CUSPATIAL_HOME
chmod +x ./build.sh
echo "NB fixed CUDA 12.6 installation. Put build into xxx/cuspatial"
./build.sh clean libcuspatial cuspatial tests '--cmake-args="-DCMAKE_INSTALL_PREFIX=xxx/cuspatial"'

Still fails at end with

no such option: --config-settings

but has compiled and built and put installation into xxx/cuspatial

Running gtests ok

cd xxx/cuspatial/cpp/build/gtests/

Run in turn each of the 54 *_TEST executable programs and check they each say their tests have PASSED (approx 1 minute in total).

Remainder of section on build.sh should only be of historical interest. However example.cu still fails to compile.

Problems with cuspatial

Section namespace describes an initial compilation problem (reported in bug report #562). This section describes problems and work arounds to build cuspatial on a Linux desktop with a NVIDIA GeForce RTX 4070 Ti SUPER (compute level 8.9) using CUDA 12.6, version 11.5.0 of the GNU gcc C++ compiler and cmake 3.26.5

git clone cuspatial or download https://github.com/rapidsai/cuspatial/archive/refs/heads/branch-24.12.zip

In our CUDA installation, we set environment variables PATH and LD_LIBRARY_PATH to point to CUDA 12.6 (You may also want to set PATH and LD_LIBRARY_PATH to use your favourite version of the GCC compiler but they were set appropriately for the most recent version of g++ so here we do not need to do that.)

Summary
   (2 Dec 2024, still incomplete:
     cuspatial and cuproj Python packages not installed?
     still get "::" compilation error on example.cu)

setenv PATH /opt/cuda/cuda-12.6/bin/:"$PATH"
setenv LD_LIBRARY_PATH /opt/cuda/cuda-12.6/lib64:"$LD_LIBRARY_PATH"

unzip cuspatial-branch-24.12.zip
mv cuspatial-branch-24.12 cuspatial
setenv CUSPATIAL_HOME `pwd`/cuspatial
setenv EXTRA_CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=~/assugi/cuproj/cuspatial -DCUDAToolkit_INCLUDE_DIR=/opt/cuda/cuda-12.6/targets/x86_64-linux/include"
setenv PARALLEL_LEVEL "-j1"
cd $CUSPATIAL_HOME && chmod +x ./build.sh && ./build.sh libcuspatial cuspatial tests

Perhaps in future we could use build.sh's --allgpuarch (build for all supported GPU architectures) to compile cuspatial on a computer without its own GPU?

build.sh cannot find cuda_runtime.h and cublas_v2.h

-- Unable to find cuda_runtime.h in "/opt/cuda/cuda-12.6/include" for CUDAToolkit_INCLUDE_DIR.
-- Unable to find cublas_v2.h in either "" or "/server/opt/math_libs/include"

Work around: define CUDAToolkit_INCLUDE_DIR

For our installation both cublas_v2.h and cuda_runtime.h are in directory /opt/cuda/cuda-12.6/targets/x86_64-linux/include and so CUDAToolkit_INCLUDE_DIR should be set to /opt/cuda/cuda-12.6/targets/x86_64-linux/include
However setting environment variable CUDAToolkit_INCLUDE_DIR outside build.sh does not work. Instead it must be set inside build.sh But as section noargs says this cannot be done via build.sh's command line. Instead it can be done via cmake's -D syntax and environment variable EXTRA_CMAKE_ARGS

setenv EXTRA_CMAKE_ARGS "-DCUDAToolkit_INCLUDE_DIR=/opt/cuda/cuda-12.6/targets/x86_64-linux/include"

build.sh fails in proj-src CMakeLists.txt

CMake Error at build/_deps/proj-src/cmake/CMakeLists.txt:110 (file):
  file RELATIVE_PATH called with incorrect number of arguments

cmake command file(RELATIVE_PATH gives an error because it needs three arguments but is given only two. On investigation, the third argument ${CMAKE_INSTALL_PREFIX} is null (hence the error).

Work around: define CMAKE_INSTALL_PREFIX

For simplicity we decided to put the built code in the same directory tree as the source code. That is we define the cmake environment variable CMAKE_INSTALL_PREFIX as ~/assugi/cuproj/cuspatial As with CUDAToolkit_INCLUDE_DIR (previous section) we did this via EXTRA_CMAKE_ARGS and cmake's -D syntax.

setenv EXTRA_CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=~/assugi/cuproj/cuspatial"

(we expanded ~ to give a complete path name).

build.sh refuses [--cmake-args=\"<args>\"]

build.sh's help says it should be possible to pass additional cmake options on the build.sh command line using --cmake-args=\"<args>\"
Paul Taylor gives an example of the correct syntax:

./build.sh clean libcuspatial cuspatial tests '--cmake-args="-DCUDAToolkit_ROOT=/opt/cuda/cuda-12.6"'

Note use of ' to enclose whole of --cmake-args=
Note use of " to enclose value for --cmake-args
Note use of cmake style -D prefix to set environment variable CUDAToolkit_ROOT inside build.sh
Note do not use []
Note do not < >

Work around: EXTRA_CMAKE_ARGS

Set the environment variable EXTRA_CMAKE_ARGS (using cmake syntax) before running build.sh build.sh will then pick up EXTRA_CMAKE_ARGS and pass its contents to cmake.

Computer crashes due to overload

The computer has 32GB ram. By default, after configuring (which takes about three minutes), cuspatial-branch-24.12/build.sh attempts to compile many CUDA C++ source files in parallel. Within a minute to two the computer is overwhelmed and becomes unusable until rebooted.

Work around: prevent cmake doing compilations in parallel

Set cmake environment variable PARALLEL_LEVEL to limit the number of jobs run in parallel. (cmake -j argument). Eg:

setenv PARALLEL_LEVEL "-j1"

${PARALLEL_LEVEL} is explicitly used by cuspatial-branch-24.12/build.sh
Note with a single thread (-j1) compilation takes about three and a half hours.

Note CMAKE_BUILD_PARALLEL_LEVEL did not work and was ignored:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_BUILD_PARALLEL_LEVEL

Apparently cmake RUN_SERIAL only applies when using cmake to run tests (not tried so far).

build.sh -v (verbose)

build.sh -v (verbose switch) was tried and does turn on cmake's verbose mode (eg it outputs nvcc compilation command lines) but so far has not proved useful for debugging build.sh

`error: name followed by "::" must be a class or namespace name`

See bug #1500 report.

nvcc compiler (v12.6) error message:

../cuspatial-branch-24.12/cpp/_deps/rmm-src/include/rmm/mr/device/device_memory_resource.hpp(312): error: name followed by "::" must be a class or namespace name
    friend void get_property(device_memory_resource const&, cuda::mr::device_accessible) noexcept {}

This is an awful error message. It seems (see forums.developer) that the problem is trying to compile cuproj code for a graphics card whose compute level does not support cuproj. (The rapidsai documentation says the minimum compute level is 6.0.)

No Work Around

But some progress, see my rapidsai/docs bug report #562

Specifying suitable compute level on nvcc command line

nvcc .. example.cu -gencode=arch=compute_60,code=sm_60

(and/or actual compute level of your GPU) did not work

W.B.Langdon 1 December 2024 (Last update 12^th Dec 2024)
back