- What do they solve?
- What is a sourcemap?
- How does adding them affect my coding?
- What are some crazy examples?
- Overcoming Limitations
Initially they were designed to map the minified JS back to the source. BUT...
mapping one text file to others doesn't just apply to JS... it's used for CSS and can be used for any text file.
In general it answers the question "Where did this text come from? (so I can edit it)"
Example: There's a broken link in a book on the web. With sourcemaps, anyone can jump straight to the line of CNXML on GitHub and fix the link.
Example: If there's a validation error in the middle of building a book, rather than output the line/column of some temporary file, sourcemaps allow the error message to refer to the line/column in the original file.
Let's start with a TypeScript file:
const foo = (name: string, age: number) => {
console.log(name, age)
}This file gets converted into JavaScript because browsers do not understand TypeScript. Here is what the file could look like:
function a0(a1,a2){console.log(a1,a2)}A SourceMap maps stuff back to the source file, in this case TypeScript.
The caret letters below (^A) are a visual representation of what is stored in a sourcemap:
// Typesscript:
const foo = (name: string, age: number) => {
// ^A ^B ^C
console.log(name, age)
// ^D ^^F ^G ^H
}
// Minified JS:
function a0(a1,a2){console.log(a1,a2)}
// ^A ^B ^C ^D ^^F ^G ^HAnd here's the same data stored in a Table:
| Description | from (JS) | to (TS) |
|---|---|---|
| foo | 1:10 | 1:7 |
| name | 1:13 | 1:14 |
| age | 1:16 | 1:28 |
| console | 1:20 | 2:5 |
| . | 1:27 | 2:12 |
| log | 1:28 | 2:13 |
| name | 1:32 | 2:17 |
| age | 1:35 | 2:23 |
- The sourcemap format is not limited to one file.
- It supports mapping one file to multiple source files.
- The spec actually maps the line+column to a file + line/column.
Consider converting a TSX file to a minified JS file. In this scenario 3 transformations occur:
- the TSX file is transformed to a JSX file
- JSX file -> JS file
- JS file -> Minified JS file
If each step generates a sourcemap file then we have a mapping from TSX -> Minified JS !
We will discuss the spec.
There are 2 parts: a valid sourcemap file and a reference to the sourcemap from the generated file (e.g. JS/CSS).
These are always a comment at the end of the file:
JS or JSON5:
...
// sourcemappingURL=minified.js.mapCSS:
...
/*# sourceMappingURL=../style.css.map */XML or HTML:
...
<!-- # sourceMappingURL=data.xml.map -->Instead of specifying the path to a separate file the sourcemap can instead be embedded in the generated JS/CSS file by using a data URI instead of a filename. A data URI is a Base64-encoded version of the sourcemap file contents.
...
// sourcemappingURL=data:text/json;base64,SGVsbG8sIFdvcmxkIQ==There are several parts to a sourcemap file:
{
"version" : 3, // (required) The version of the SourceMap Specification
"file": "out.js", // (optional) The generated code that this map is associated with
"sourceRoot": "", // (optional) A path perfix for all the files named in "sources"
"sources": ["foo.tsx", "bar.tsx"], // (required) A list of the source files, referred to in the "mappings" section
"sourcesContent": ["import ...", null], // (optional) The content of the source files
"names": ["firstName", "lastName", "i"],// (optional) List of symbol names used by the "mappings" section
"mappings": "A,AAgBC;;ABCDE;" // (required) A Base64 VLQ that contains the mappings
}list of files, maps, optional source code (base64 encoded)
Let's start with a decoded mappings section and build up to the encoded one.
{
version: 3,
sources: [ "foo.tsx", "bar.tsx" ]
// Decoded:
mappings: [
{ generatedLine: 0, generatedColumn: 0, sourceLine: 16, sourceColumn: 1, sourceFile: 0 },
{ generatedLine: 0, generatedColumn: 1, sourceLine: 0, sourceColumn: 0, sourceFile: 1 },
]
// Encoded:
mappings: "AAgBC;AB"
}Here's a code snippet that we can run in Runkit:
var {SourceMapGenerator} = require("source-map")
const g = new SourceMapGenerator()
g.addMapping({
source: 'foo.tsx',
original: { line: 16 + 1, column: 1 },
generated: { line: 0 + 1, column: 0 },
})
g.addMapping({
source: 'bar.tsx',
original: { line: 0 + 1, column: 1 },
generated: { line: 1 + 1, column: 0 },
})
g.toJSON()
// {version: 3, sources: ["foo.tsx", "bar.tsx"], names: [], mappings: "AAgBC;AChBA"}The mappings section looks like this:
- The mappings section is a bunch of groups separated by
; - Each group contains segments separated by
, - Each segment is made up of 1,4, or 5 variable length fields
Each segment is Base64 encoded
Most mappings are near each other so instead of storing every line/column number all the time (1234, 1235, 1233) the spec just stores the offset (1234, +1, -2).
http://npmjs.com/packages/source-map
It's a few steps that can be hidden behind a library:
- Annotate the parsed text file (XML/JSON) with source line/column numbers
- With XML/HTML files folks will typically manipulate Document Nodes... each Node just needs to keep the source line/column
- With JSON, see json-c parser
- (...do normal work moving these objects around in memory...)
- Write the output file out along with the sourcemap file
- or embed the sourcemap inside the file
This experiment converts XML Config files into CSS. See it by opening this link in Chrome and inspecting the elements.
Or, if it does not work for you, check out this screencap to see the process.
The big one is that libxml can provide line information about nodes but not column information.
Whats worse is that most languages just slap a coat of paint on top of libxml.
2 languages that we use don't do that: JavaScript & Java
But there's a way to address this: have a step before that formats the XML so that every element is on a separate line. Example:
<p id="a" class="b">Hello !<b>You</b></p>
<!-- Changes to something lie this -->
<p id="a" class="b"
>Hello !<b
>You</b
></p
>