Hashes have a number of overlapping but not entirely redundant or
equivalent purposes within the build system.
Hashes of source code are important because they can be computed before
compilation and processing, and thus are useful as keys for caching that
expensive work. Source hashes remain useful even after compilation, as a
way of reflecting the contributions of source-code-sensitive assets like
source maps.
However, source hashes do not tell the whole story, and using them as
cache keys can be risky if the work that's being cached depends on
generated code rather than source code, as we recently discovered with the
findImportedModuleIdentifiers function. The preliminary fix for that
problem (#10330) was to cache findImportedModuleIdentifiers using a hash
of the generated code rather than the source hash.
PR #10330 swung a bit too far in the direction of ignoring source hashes
and considering only hashes of generated code. For example, the URLs of
source maps share the hash of the corresponding resource, but source maps
can change (because of superficial changes in the source code) without
changing the generated code of the resource. Ignoring the source hash when
computing source map URLs resulted in stale source maps with incorrect
line numbers.
A better solution seems to be to propagate the source hash (along with any
hashes of intermediate generated artifacts) all the way through bundling,
so that the final hash of any static resource reflects all information
that could/should change the behavior of that static resource, including
its source map, which embeds the exact source code of all contributing
files in the sourcesContent property. At every step of the way, we merge
all the input hashes into a single hash, so we don't have to keep juggling
multiple hashes, thankfully.
Sub-Resource Integrity (SRI) hashes still need to be computed from just
the final contents of a given asset, so that the browser can verify those
contents without knowing anything about the Meteor build system, but
that's handled separately.
Best I can tell, the major version portion of Chromium versions has always
tracked all the way through to Chrome Canary, Dev and Stable releases.
Since we observe the major version of Chrome in terms of identifying it as a
"modern browser", it seems to make sense to treat "Headless Chrome" and
Chromium in the same regard.
Interestingly, when the same Chrome as we all use on our machines is run
with the `--headless` flag, it switches its `navigator.userAgent` to
`HeadlessChrome/`, rather than `Chrome/`.
This was initially problematic since the `useragent` npm we use for parsing
user agents didn't understand this designation, however, with the update of
`webapp`'s `useragent` npm in 058351b7, `headlesschrome` will now have its
version available from `WebAppInternals.identifyBrowser`, so we can
accurately identify it and serve it the modern bundle.
Previously, while the `useragent` package was able to parse the User-Agent
for so-called "Headless Chrome" and generate a family of "HeadlessChrome",
it was unable to parse out the individual portions of the version number
(e.g. major, minor, patch).
For example, the following User-Agent (herein referred to as `userAgentAbove`):
```
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.100 Safari/537.36
```
Previously resulted in:
```
> require('useragent').lookup(userAgentAbove);
{
family: 'HeadlessChrome',
major: '0',
minor: '0',
patch: '0',
/* ... */
}
```
With the newer version of `useragent`, these are now properly extracted and
set which will enable Meteor to treat Headless Chrome the same as Chrome in
a follow-up commit. Now:
```
> require('useragent').lookup(userAgentAbove);
{
family: 'HeadlessChrome',
major: '69',
minor: '0',
patch: '3497'
/* ... */
}
```
Best I can tell, the major version portion of Chromium versions has always
tracked all the way through to Chrome Canary, Dev and Stable releases.
Since we observe the major version of Chrome in terms of identifying it as a
"modern browser", it seems to make sense to treat "Headless Chrome" and
Chromium in the same regard.
Interestingly, when the same Chrome as we all use on our machines is run
with the `--headless` flag, it switches its `navigator.userAgent` to
`HeadlessChrome/`, rather than `Chrome/`.
This was initially problematic since the `useragent` npm we use for parsing
user agents didn't understand this designation, however, with the update of
`webapp`'s `useragent` npm in 058351b7, `headlesschrome` will now have its
version available from `WebAppInternals.identifyBrowser`, so we can
accurately identify it and serve it the modern bundle.
Previously, while the `useragent` package was able to parse the User-Agent
for so-called "Headless Chrome" and generate a family of "HeadlessChrome",
it was unable to parse out the individual portions of the version number
(e.g. major, minor, patch).
For example, the following User-Agent (herein referred to as `userAgentAbove`):
```
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.100 Safari/537.36
```
Previously resulted in:
```
> require('useragent').lookup(userAgentAbove);
{
family: 'HeadlessChrome',
major: '0',
minor: '0',
patch: '0',
/* ... */
}
```
With the newer version of `useragent`, these are now properly extracted and
set which will enable Meteor to treat Headless Chrome the same as Chrome in
a follow-up commit. Now:
```
> require('useragent').lookup(userAgentAbove);
{
family: 'HeadlessChrome',
major: '69',
minor: '0',
patch: '3497'
/* ... */
}
```
After @nathan-muir's PR #10053, we did not publish a new version of the
diff-sequence package, which would have contained DiffSequence.diffMaps.
I honestly have no idea why #10320 did not manifest before now, but
publishing these changes seems to fix it.
With the introduction of lazy compilation in Meteor 1.8, calling
inputFile.addJavaScript({
...
hash: inputFile.getSourceHash(),
...
}, function () {
return compiler.processFilesForTarget(inputFile);
});
becomes problematic, since inputFile.getSourceHash() is usually different
from compiler.processFilesForTarget(inputFile).hash, because the latter is
computed from the compiled code, whereas the former is computed from the
source code.
For example, when we use file.hash to cache imported module identifiers in
ImportScanner#_findImportedModuleIdentifiers, we really need to be using
the hash of the compiled code, since a single source module can be
compiled in different ways. If we cache based on the source hash, there's
a risk of reusing the scanned imports from the web.browser version for the
web.browser.legacy version, which can lead to all sorts of problems that
are only apparent in legacy browsers.
The quick fix is easy enough: BabelCompiler can simply stop including a
hash in the eager options to inputFile.addJavaScript. This fix can be
published as a minor update to the babel-compiler and ecmascript packages.
The remaining changes in this commit add another layer of defense against
this problem, by ignoring any hash options provided by compiler plugins,
in favor of simply computing the hash from the compiled data buffer.
These additional changes will become available in the next release of
Meteor (likely 1.8.1).
