.. _sec-ref-ospray:

OSPRay
======

OSPRay is Intel's Open Source Portable Ray tracing engine,
applied in data visualisation and (interactive) rendering.
The engine relies on the `Intel Embree 
<https://www.embree.org/>`_ kernels and the `Intel SPMD 
Progam Compiler <https://ispc.github.io/index.html>`_.
Documentation can be found on the `OSPRay wbsite 
<https://www.ospray.org/documentation.html>`_ and on the 
`OSPRay GitHub <https://github.com/ospray/OSPRay>`_.

Installing OSPRay
-----------------

The following is a step-by-step list to install OSPRay on the system
Note that this will install OSPRay without graphical interfaces,
such as the Exampleviewer. The NextgenioIO system currently does 
not have the graphical libraries installed to enable these features.

1. Load the required modules

   .. code:: bash

      module load cmake embree glfw mesa


2. Clone the OSPRay repository

   .. code:: bash

      git clone https://github.com/ospray/ospray.git 

   This will create and fill a directory named *ospray* in the
   current working directory.

3. Create a build directory in the repository directory

   .. code:: bash

      cd ospray
      mkdir build
 
   Creating a separate build directory is generally helpful when 
   multiple configurations of the build are possible or required.

4. Create an install directory, where the executables and header files
   will be installed.

5. Configure the CMake file in the build directory. As the GUI
   CCMake is included in the *cmake* module, it is recommended to make 
   use of it, as there are multiple options to set, and entering them all
   in a single command line argument may be troublesome.

   .. note::

      For some of the options it may be necessary to request a CCMake
      configuration (hit *c*) to be prompted with the necessary command 
      options. Warning messages can be exited by hitting *e* to return 
      to the configuration menu. Other options (such as switching on MPI)
      can only be entered by loading the full menu of options by hitting *t*
      in the settings menu.

      The CCMake menu can be searced by entering '/' followed by the search
      string.

      Not all options will have to be set: this is not necessary for a 
      working install of OSPRay.

   a. Enter the build directory and start the GUI

      .. code:: bash

         cd build
         ccmake ..

      This will load the CMakeLists.txt file in the repo directory

   b. Start initial configuration: hit *c*

   c. There are many settings to controll the build of the application
      (hit *t* to switch the full list on/off). Below is a list of the 
      settings that will likely need to be altered and are necessary for
      a successful install on the system.

      ============================ =================================
      CMAKE_BUILD_TYPE              Release
      CMAKE_INSTALL_PREFIX          /path/to/install/directory
      OSPRAY_APPS_BENCHMARK         ON
      OSPRAY_ENABLE_TUTORIALS       OFF
      OSPRAY_APPS_EXAMPLEVIEWER     OFF
      OSPRAY_MODULE_MPI             ON
      BUILD_SHARED_LIBS             ON
      ISPC_EXECUTABLE               /path/to/ispc-v1.9.1-linux/ispc
      ============================ =================================

      OSPRay uses TBB as its threading system by default (which is 
      installed on the NextgenIO system), but can also use OpenMP.
      To enable this option select:

      ============================ ==========
      OSPRAY_TASKING_SYSTEM         OpenMP
      ============================ ==========

   d. In case not all the required libraries and/or header files
      are found by CMake, an error message will be shown (exit with
      *e*) and the path to these files will need to be set manually.

      An overview of possibly problematic libraries and their 
      required setting (see also the warning at the end of this
      section).

      ======================= =======================================================
      glfw3_DIR                /home/software/glfw/3.3
      OPENGL_EGL_INCLUDE_DIR   /home/software/mesa/18.0.0/include/EGL
      OPENGL_GLX_INCLUDE_DIR   /home/software/mesa/18.0.0/include/GL
      OPENGL_INCLUDE_DIR       /home/software/mesa/18.0.0/include/GL
      OPENGL_egl_LIBRARY       /home/software/mesa/18.0.0/lib/libEGL.so
      OPENGL_gl_LIBRARY        /home/software/mesa/18.0.0/lib/libGL.so
      OPENGL_opengl_LIBRARY    /path/of/local/copy/lib/libOpenGL.so
      ======================= =======================================================

   e. After the final configuration (hitting *c* without being prompted for
      further input) the Makefile can be generated by hitting *g*.

      CCMake will most likely show a warning concerning the inability to 
      guarantee a safe runtime search path. This is the result of some libraries
      being installed twice as part of different modules and will not affect the 
      ability to run OSPRay after building and installing. 

      Upon completion the Makefile should be present in the *build* directory.

6. Build OSPRay

   .. code:: bash

      make

   The build may take considerable time

7. Install OSpray

   .. code:: bash

      make install

   This will install the executable and header files in the
   install directory specified in step 5.


8. Update LD_LIBRARY_PATH

   Add the build path to LD_LIBRARY_PATH (note: linking to the
   install path may lead to the error :ref:`ref-ospray_mpiinit` 
   when trying to use MPI):

   .. code:: bash

      export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/install/directory/lib64

   It can also be useful to add the *bin* directory
   to the PATH to be able to run the executables without 
   having to specify the full path:

   .. code:: bash

      export PATH=$PATH:/path/to/install/directory/bin
      
   By adding these lines to the .bashrc file these variables
   will be set automatically when logging into a shell.


9. **Optional**: Include MAP MPI wrappers in ospBenchmark

   To enable MAP profiling of the benchmarking application included
   in the OSPRay build, two MAP libraries need to be linked to the
   *ospBenchmark* executable (Note: this assumes the CMake variable 
   OSPRAY_APPS_BENCHMARK is the to ON). Instructions on how to build
   the MAP libraries can be found in the section :ref:`sec-ref-maplib`.

   The libraries need to be linked by modifying one of the commands in 
   the makefile. This requires some small modifications to the step 
   outlined about above:

   a. Set the CMake variable 

      ============================ ==========
       CMAKE_VERBOSE_MAKEFILE       ON
      ============================ ==========

   b. Save the output of running ``make``:

      .. code:: bash

         make > make_output

   c. Create the MPI Wrapper libraries in the build directory

      .. code:: bash

         make-profiler-libraries

   d. Search for the relevant command, which links the libraries and 
      object code for *ospBenchmark*:

      .. code:: bash

         cat make_output | grep ospBenchmark

      The required part of the output will resemble:

      ::    

         [ 88%] Linking CXX executable ../../ospBenchmark

         cd /path/to/build/directory/apps/bench && 
         /home/software/cmake/3.14.4/bin/cmake -E cmake_link_script CMakeFiles/ospBenchmark.dir/link.txt --verbose=1
         /opt/intel/compilers_and_libraries_2019.3.199/linux/bin/intel64/icpc  -static-intel -Wall -fPIC -fno-strict-aliasing 
         -no-ansi-alias -DNOMINMAX -Wno-unknown-pragmas  -DNDEBUG -O3  -rdynamic CMakeFiles/ospBenchmark.dir/bench.cpp.o  
         -o ../../ospBenchmark -Wl,-rpath,/path/to/build/directory: ../../libospray_app.a 
         ../../libospray_sg.so.1.8.5 ../../libospray.so.1.8.5 ../../libospray_tfn.so.1.8.5 ../../libospray_json.so.1.8.5 
         ../../libospray_common.so.1.8.5 -pthread /opt/intel/compilers_and_libraries_2019.3.199/linux/tbb/lib/intel64/gcc4.7/libtbb.so.2 
         /opt/intel/compilers_and_libraries_2019.3.199/linux/tbb/lib/intel64/gcc4.7/libtbbmalloc.so.2 
         -ldl

   e. Modify the command and enter (the added lines are marked with a (*) ):

      .. code:: bash

         cd /path/to/build/directory/apps/bench && 
         /home/software/cmake/3.14.4/bin/cmake -E cmake_link_script CMakeFiles/ospBenchmark.dir/link.txt --verbose=1
         /opt/intel/compilers_and_libraries_2019.3.199/linux/bin/intel64/icpc  -static-intel -Wall -fPIC -fno-strict-aliasing 
         -no-ansi-alias -DNOMINMAX -Wno-unknown-pragmas  -DNDEBUG -O3  -rdynamic CMakeFiles/ospBenchmark.dir/bench.cpp.o  
         -o ../../ospBenchmark -Wl,-rpath,/path/to/build/directory: ../../libospray_app.a 
         ../../libospray_sg.so.1.8.5 ../../libospray.so.1.8.5 ../../libospray_tfn.so.1.8.5 ../../libospray_json.so.1.8.5 
         ../../libospray_common.so.1.8.5 -pthread /opt/intel/compilers_and_libraries_2019.3.199/linux/tbb/lib/intel64/gcc4.7/libtbb.so.2          /opt/intel/compilers_and_libraries_2019.3.199/linux/tbb/lib/intel64/gcc4.7/libtbbmalloc.so.2 
         (*) /path/to/build/directory/libmap-sampler.so
         (*) /path/to/build/directory/libmap-sampler-pmpi.so
         -ldl

   f. Install OSPRay (return to the main build directory before running
      this command)

      .. code:: bash

         make install

   g. After running ``make install`` the two libraries need to be visible
      to the *ospBenchmark** application. This can be achieved either by 
      including the location in LD_LIBRARY_PATH or by copyng the wrappers
      to the install location of the OSPRay libraries. To copy the libraries 
      (assuming they were built in the OSPRay build directory):

      .. code:: bash

         cp /path/to/build/directory/libmap* /path/to/install/directory/lib64

10. **Optional**: Enabling the Open Image Denoiser

   This is an extra option, which is not necessary for an operational
   build of OSPRay, but could improve performance (*not tested*). This step
   needs to be included before step 5. is completed. It will require
   the download of a separate executable and linking to its directory.

   a. Download Intel Open Image Denoiser

      .. code:: bash 

         wget https://github.com/OpenImageDenoise/oidn/releases/download/v0.9.0/oidn-0.9.0.x86_64.linux.tar.gz

         tar -xzf oidn-0.9.0.x86_64.linux.tar.gz

   b. Set the CMake variable      

      ============================ ==========
      OSPRAY_APPS_ENABLE_DENOISER   ON
      ============================ ==========

   c. Set the path to the OIDN (it may be necessary to configure
      within CCMake -hit *c*- first).

      The directory path should point to the location of
      OpenImageDenoiseConfig.cmake:

      ============================ ========================================================
      OpenImageDenoise_DIR          /path/to/oidn-0.9.0.x86_64/lib/cmake/OpenImageDenoise/ 
      ============================ ========================================================




.. warning:: Known issues with OSPRay install

   - Missing libraries: libOpenGL.so

     Workaround: create a directory with a local copy of the library
     
     A copy of the library can be found in the Debian package
     *libopengl0*, which can then be copied onto the system 

     A more permanent solution is likely desired.

   - ISPC version (1.11)

     This will result in:

     .. error::

        relocation R_X86_64_PC32 against symbol 'stdout@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
     
     Workaround: Create a local install of ISPCv1.9

     A pre-built version of the compiler is available for download.
     Below are the instructions to install ISPC in a separate directory:

     .. code:: bash

        wget https://downloads.sourceforge.net/project/ispcmirror/v1.9.1/ispc-v1.9.1-linux.tar.gz --no-check-certificate
     
        tar -xzf ispc-v1.9.1-linux.tar.gz

     The ISPC executable is located in the main directory (ispc-v1.9.1-linux). 
     The CMake variable ISPC_EXECUTABLE should list the directory path
     and the executable.

     *Updates may be required if the modules are changed*