Skip to content

Instantly share code, notes, and snippets.

@jxramos
Last active March 8, 2022 19:11
Show Gist options
  • Save jxramos/37f87b40f796c6f4ee5986c4d2086c03 to your computer and use it in GitHub Desktop.
Save jxramos/37f87b40f796c6f4ee5986c4d2086c03 to your computer and use it in GitHub Desktop.
Documenting any cool bazel stuff found along the way

https://github.com/bazelbuild/starlark/

which links to

https://github.com/bazelbuild/starlark/blob/master/spec.md

This document was derived from the [description of the Go implementation](description of the Go implementation) of Starlark.

Which explains why all the buildifier and buildozer stuff is expressed in Go. It was a fallacy to think that the starlark interperter would be in python like the language representation itself.

Searching for starlark imports in the buildtools go files lands upon this very promising beauty...

//  ...use the AST from go.starlark.net/syntax. This will give
// us a much more precise AST and will allow us to share code with the
// Skylark interpreter. The end goal is to build a number of tools able
// to parse, analyze, format, refactor, evaluate Skylark code.

https://github.com/bazelbuild/buildtools/blob/master/buildifier2/buildifier2.go#L6-L10

Bazel main source most likely https://github.com/bazelbuild/bazel/blob/bbea57557a80716ae09aaef62a1a7f0b9f097ceb/src/main/java/com/google/devtools/build/lib/bazel/Bazel.java#L73-L76

The bazel build file tools https://stackoverflow.com/questions/58598412/are-the-bazel-buildtools-primarily-focused-on-single-starlark-files

What bazel produces during a build, several symlink folders under the workspace per workspace. https://docs.bazel.build/versions/master/output_directories.html

Discussion on Bazel design choices https://stackoverflow.com/questions/29245787/what-are-the-differences-between-bazel-and-gradle?rq=1

Bazel query supported native output formats https://docs.bazel.build/versions/master/query.html#output-formats

Bazel Output Formats

Actual output rendering occurs at this stage

GenQueryConfiguration genQueryConfig =
    ruleContext.getConfiguration().getFragment(GenQueryConfiguration.class);
GenQueryOutputStream outputStream =
    new GenQueryOutputStream(genQueryConfig.inMemoryCompressionEnabled());
try {
  QueryOutputUtils
      .output(queryOptions, queryResult, targets.getResult(), formatter, outputStream,
      queryOptions.aspectDeps.createResolver(packageProvider, getEventHandler(ruleContext)));
  outputStream.close();

https://github.com/bazelbuild/bazel/blob/c7d2b8d2c8c5f71b6a8d7af0190a0500266bf12b/src/main/java/com/google/devtools/build/lib/rules/genquery/GenQuery.java#L388

The particular output formatter drawn from the command line occurs here

  formatter =
      OutputFormatters.getFormatter(
          OutputFormatters.getDefaultFormatters(), queryOptions.outputFormat);

https://github.com/bazelbuild/bazel/blob/c7d2b8d2c8c5f71b6a8d7af0190a0500266bf12b/src/main/java/com/google/devtools/build/lib/rules/genquery/GenQuery.java#L333-L335

We'd need to add a new output formatter

https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/query2/query/output/GraphOutputFormatter.java

The starlark AST module statements have a From and To set of fields that follow the aliasing support for imports. The From is the right hand side of the assignments, the symbols being loaded, the left identifiers are under the To dict, where these symbols are being aliased to.

Eg

load("//my_bazel_source.bzl", "var1", var_new="var2")

will expand to

        {
            "Load": {
                "Line": 3,
                "Col": 1
            },
            "Module": {
                "Token": 8,
                "TokenPos": {
                    "Line": 3,
                    "Col": 6
                },
                "Raw": "\"//example//:source.bzl\"",
                "Value": "//example//:source.bzl"
            },
            "From": [
                {
                    "NamePos": {
                        "Line": 3,
                        "Col": 80
                    },
                    "Name": "var1",
                    "Binding": null
                },
                {
                    "NamePos": {
                        "Line": 3,
                        "Col": 90
                    },
                    "Name": "var2",
                    "Binding": null
                }
            ],
            "To": [
                {
                    "NamePos": {
                        "Line": 3,
                        "Col": 68
                    },
                    "Name": "var1",
                    "Binding": null
                },
                {
                    "NamePos": {
                        "Line": 3,
                        "Col": 68
                    },
                    "Name": "var_new",
                    "Binding": null
                }
            ],
            "Rparen": {
                "Line": 3,
                "Col": 80
            }
        },

Starlark AST has arguments as None upon parsing void parameter expressions such as declare_foo()

#Writing rules https://jayconrod.com/posts/106/writing-bazel-rules--simple-binary-rule

Compilation Databases

https://www.cppdepend.com/GettingStartedForLinux

...other ways to generate a compilation database as described here. https://sarcasm.github.io/notes/dev/compilation-database.html#how-to-generate-a-json-compilation-database https://sarcasm.github.io/notes/dev/compilation-database.html#bazel

https://kythe.io/getting-started/ kythe/kythe#4067

That script (generate_compilation_database.sh) isn't really particularly well maintained and will really only work when generating a compilation database for Kythe itself.

bazelbuild/bazel#258 obtain compilation flags or compilation database from bazel #258 great discussion where official support for exporting a compilation database is directly mentioned in the bazel repo.

https://github.com/grailbio/bazel-compilation-database comments on alternatives

These approaches (kythe) could be more accurate than the approach of this tool in some rare cases, but need a more complicated setup and a full build every time you refresh the database.

https://github.com/aszinovyev/bazel-compilation-db derived from kythe, instructions for integrating into your WORKSACE and copying a folder to your repo to be built.

https://gist.github.com/bsilver8192/0115ee5d040bb601e3b7

https://github.com/vincent-picaud/Bazel_and_CompileCommands

https://docs.bazel.build/versions/main/build-event-protocol.html

https://docs.bazel.build/versions/main/be/make-variables.html#predefined_label_variables

https://docs.bazel.build/versions/main/test-encyclopedia.html#initial-conditions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment