30 Project Setup

Fangrui Song edited this page 2023-12-16 13:36:22 -08:00

Table of Contents

• How ccls obtains sources and compilation commands
• Using build containers
• ccls on the host
• ccls on the container
• compile_commands.json
• CMake
• Build EAR
• compiledb
• scan-build
• Ninja
• Meson
• GN
• Waf
• buck
• xcodebuild
• stdout of an external command
• .ccls File
• %compile_commands.json
• %c / %cpp / %objective-c / %objective-cpp
• %cu
• %h / %hpp
• Compiler driver
• .ccls examples
• Example A
• Example B
• Example: -march=armv7a
• IAR Embedded Workbench for Arm

ccls typically indexes an entire project. In order for this to work properly, ccls needs to be able to obtain the source file list and their compilation command lines.

How ccls obtains sources and compilation commands

There are two main ways this happens:

1. Provide compile_commands.json at the project root
2. Provide a .ccls file. It is a line-based text file describing compiler flags. Recursively listed source files (headers excluded) will be indexed. Use an empty .ccls to get started (if you don't need specific -I, -D, etc).

If neither exists, then when ccls starts it will not index anything: instead it will wait for LSP clients to open files and index only those files.

Using build containers

Build containers are usually OCI containers and are often managed by Podman or Docker. Toolchains for building might run in these containers for reproducability.

There are two possibilities:

1. Execute the container for building only. Use ccls and your editor/IDE on the host.
2. The build container is running in the background with an installed and running ccls. Furthermore, it is using the host network so that editors/IDEs running on the host can connect.

ccls on the host

If you want to use ccls on the host, a path mapping has to be applied, since paths and compiler versions will most probably differ. Thus, paths of following headers have to be translated to the host system:

• system and third-party libraries (if those headers do not already exist, install them. E.g., openssl-devel in Linux)
• the compiler's headers

See following initialization options (ccls --init='...'), which uses build/compile_commands.json already produced in the container:

{
  "clang": {
    "resourceDir": "/usr/lib64/clang/13.0.0",
    "pathMappings": [
      "/project_container/>/project_host/",
      "/opt/rh/gcc-toolset-10/root>"
    ],
    "extraArgs": [
      "--gcc-toolchain=/opt/rh/gcc-toolset-10/root/usr"
    ]
  },
  "compilationDatabaseDirectory": "/project_host/build/"
}

resourceDir denotes the directory of the compiler's include files at the host. The compiler version is usually different from the container's. pathMappings substitute directories mounted into the container by the related host directories. If you are using clang(++) with --gcc-toolchain=..., then you have to add the corresponding directory to pathMappings, and add clang.extraArgs. Here the compile_commands.json is in a different directory, and, hence, compilationDatabaseDirectory has to be specified.

Note: On the host, you may have to install additional developer packages (e.g. openssh-devel), which include required headers. If headers are missing, you will see errors in the log file (start ccls by --log-file=/tmp/ccls.log -v=1) or error labels in your editor.

ccls on the container

1. Install ccls in the container
2. Execute the container with --network=host (Podman, Docker) such that it continues to run.
3. Start the build process by attaching to the contatiner and executing the build tool (cmake, make,...)

compile_commands.json

Guillaume Papin(@Sarcasm) has a thorough article about compilation databases.

Generally this file is not checked into source control, but rather is generated by the build system. This means it's best to generate it in a new project before starting the ccls server in that project.

Because this file is auto-generated it's not easy to customize. As a result it's possible to provide both compile_commands.json and .ccls in the same project and have the .ccls configuration enhance the options from compile_commands.json.

If your compile_commands.json is not kept in the project root, set the initialization option compilationDatabaseDirectory to an alternative directory containing compile_commands.json.

You can use jq to concatenate multiple compile_commands.1.json fragments:

cat a/compile_commands.json b/compile_commands.json | jq '.[]' | jq -s '.' > compile_commands.json

CMake

% cmake -H. -BDebug -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=YES
% ln -s Debug/compile_commands.json .

Caveat on Windows: CMake dumps Windows shell command line directly into command, depends on how ccls was built, it may confuse this field as command line from a POSIX shell, in which Windows path separator '\' is a escape character. You can use jq to convert such entries to use arguments which does not have this issue:

jq '[.[] | {directory: .directory, file: .file, arguments: .command | split(" ") | map(select(length > 0)) | map(sub("\\\\\""; "\""; "g"))}]' < compile_commands.json

In some setups, CMake also puts source file of a translation unit (i.e. *.cpp file) after two dashes --, which means that after it no command-line options can be given. Unfortunately, ccls currently tries to add them just to the end. This modification of above helps:

jq '[.[] | {directory: .directory, file: .file, arguments: .command | split(" ") | map(select(length > 0 and . != "--")) | map(sub("\\\\\""; "\""; "g"))}]' < compile_commands.json

Build EAR

Bear is a tool that generates a compilation database for clang tooling. It can be used for any project based on Makefile.

bear -- make
# generates compile_commands.json

compiledb

scan-build

scan-build is a python package that can generate a compilation database for clang tooling (uses Bear as a backend). This too can be used for any project based on a Makefile.

intercept-build make all # generates compile_commands.json from the `make all` ruleset

Ninja

# Format: ninja -t compdb rule_names... > compile_commands.json
ninja -C out/Release -t compdb cxx cc > compile_commands.json

Meson

meson build # generates compile_commands.json in the `build` directory

GN

# Format: gn gen out/Release --export-compile-commands[=<target_name1,target_name2...>]
gn gen out/Release --export-compile-commands=cc

Waf

Load the clang_compilation_database tool in your wscript:

def configure(conf):
    conf.load('clang_compilation_database')

./waf configure build
ln -s build/compile_commands.json

buck

buck build :helloworld#compilation-database
ln -s $(buck targets --show-output :helloworld#compilation-database | cut -d ' ' -f 2)

xcodebuild

xcpretty is a 3rd party tool that can parse the output of xcodebuild and generate a compile_commands.

xcodebuild | xcpretty -r json-compilation-database --output compile_commands.json

stdout of an external command

If the initialization option "compilationDatabaseCommand" is set, the command will be executed by ccls to provide the JSON compilation database. ccls will read its stdout rather than read compile_commands.json. This may be useful when ccls cannot parse the compile_commands.json correctly (e.g. MSVC cl.exe, Intel C++ Compiler options)

ccls shell script wrapper:

#!/bin/zsh
/path/to/Release/ccls --init='{"compilationDatabaseCommand":"/tmp/c/x"}' "$@"

/tmp/c/x:

#!/bin/zsh
# cat >> /tmp/initialization-options # stdin is initialization options
print '[{"arguments":["c++","-c","a.cc"],"directory":"/tmp/c","file":"a.cc"}]'

Suppose the project is at /tmp/c, /tmp/c/x /tmp/c will be executed with stdin=initializationOptions and the stdout should be a JSON compilation database.

An example to scrub Intel C++ Compiler options (or, even easier, check out clang.excludeArgs in the Initialization options):

#!/usr/bin/env python3
import json
import os
import sys
with open(os.path.join(sys.argv[1], 'compile_commands.json')) as f:
    db = json.load(f)
    for entry in db:
        args = entry['arguments']
        try:
            # Intel C++ Compiler option that is unknown to clang
            args.remove('-xHost')
        except ValueError:
            pass
    json.dump(db, sys.stdout)

.ccls File

.ccls is a line-based text file at the project root. Its specifies compiler flags needed to properly index your code: -I -D etc. The first line specifies the compiler driver (usually clang), while each subsequent lines specifies one argument to be added to the compiler command line. Blank and "# comment" lines are ignored.

No whitespace splitting is performed on the argument, thus -I foo cannot be used (use -Ifoo or -I\nfoo for example). If an include path has spaces enter that as without escaping the spaces. For example, -I/include/path/with a space/to/dir/foo.

Subdirectories of the project can also contain .ccls files, if needed, to specify compiler flags specific to those directories.

A line may optionally start with one or more % directives, which specialize the argument on that line.

Available directives include:

%compile_commands.json

By default .ccls compiler flags are applied only to files not listed in compile_commands.json. If this directive appears first in .ccls, the compiler driver must be omitted. After compile_commands.json is parsed, the rest of the .ccls arguments will be appended to the compiler flags for files found in compile_commands.json.

%c / %cpp / %objective-c / %objective-cpp

This argument should be added only when parsing C (%c), C++ (%cpp), Objective-C (%objective-c), or Objective-C++ (%objective-c++) files.

%cu

This argument should be added only when parsing CUDA files. If you want an option to be added to both CUDA and regular C++ files, write %cpp %cu -DA.

%h / %hpp

This argument should be added only when indexing C header files (%h: *.h) or C++ header files (%hpp: *.hh *.hpp). Note, *.h files are considered as C, not C++.

You may add these lines to make every *.h parsed as C++:

%h -x
%h c++-header

Note, if your project has both C and C++ files, a.h's flags may be inferred from a C file and thus parsed as C. You may run into parsing errors like unknown type name 'class'.

Compiler driver

Unless %compile_commands.json is used, you must specify a "compiler driver" as the first line of your .ccls file.

• clang defaults to GCCMode (gcc). It correctly treats .c files as C and .cpp files as C++.
• clang++ defaults to GXXMode (g++). It treats a .c file as C++ and issues a warning.

clang a.cc and clang++ a.cc are different, but the difference is only related to linking (what default runtime libraries are passed) and is not relevant for the frontend actions ccls performs. So for ccls use cases, just stick with clang.

.ccls examples

Example A

clang
%c -std=c11
%cpp -std=c++2a
%h %hpp --include=Global.h
-Iinc
-DMACRO

*.h *.hh *.hpp files will be parsed with extra --include=Global.h

Example B

%compile_commands.json
%c -std=c11
%cpp -std=c++14
%c %cpp -pthread
%h %hpp --include=Global.h
-Iinc

It appends flags so clang should not be used.

Example: -march=armv7a

See https://github.com/MaskRay/ccls/issues/107. If the compiler driver is a GCC cross-compiler, --target= may be required. Suppose arm-linux-gnueabi-gcc -march=armv7a is used, add a --target=:

%compile_commands.json
--target=armv7a-linux-gnueabi

Otherwise clang will error: unknown target CPU 'armv7a'.

IAR Embedded Workbench for Arm

See https://github.com/MaskRay/ccls/issues/476.

%compile_commands.json
-target
armv7-linux-gnueabi
-D__ICCARM__
-U__GNUC__
-U__clang__
-isystem../Config/exec/arm/inc/c

Home

• Build
• Install
• Editor Configuration
  • Atom
  • Emacs
    • lsp-mode (+ emacs-ccls)
    • eglot
  • Kakoune
  • Sublime
  • Vim/Neovim
    • ALE
    • coc.nvim
    • LanguageClient-neovim
    • nvim-lspconfig
    • vim-lsp
    • vim-lsc
    • YouCompleteMe
  • Visual Studio Code
  • Monaco Editor
• Project Setup
  • Example Projects
• Customization
• Debugging
• FAQ
• LSP Extensions
• LSP