Understanding the Gemfile.lock file

After running the bundle install command, 'Gemfile.lock' is created in the working directory. What do the directives inside that file mean?

For example, let's take the following file:

PATH
  remote: .
  specs:
    gem_one (0.0.1)

GEM
  remote: http://example.org/
  specs:
    gem_two (0.0.2)
    gem_three (0.0.3)
      gem_four (0.0.4)

PLATFORMS
  platform

DEPENDENCIES
  gem_two
  gem_one!

What do 'PATH', 'GEM', 'PLATFORMS' and 'DEPENDENCIES' describe? Are all of them required?

What should contain the 'remote' and 'specs' subdirectives?

What does the exclamation mark after the gem name in the 'DEPENDENCIES' group mean?

Solution 1:

You can find more about it in the bundler website (emphasis added below for your convenience):

After developing your application for a while, check in the application together with the Gemfile and Gemfile.lock snapshot. Now, your repository has a record of the exact versions of all of the gems that you used the last time you know for sure that the application worked...

This is important: the Gemfile.lock makes your application a single package of both your own code and the third-party code it ran the last time you know for sure that everything worked. Specifying exact versions of the third-party code you depend on in your Gemfile would not provide the same guarantee, because gems usually declare a range of versions for their dependencies.

Solution 2:

I've spent the last few months messing around with Gemfiles and Gemfile.locks a lot whilst building an automated dependency update tool¹. The below is far from definitive, but it's a good starting point for understanding the Gemfile.lock format. You might also want to check out the source code for Bundler's lockfile parser.

You'll find the following headings in a lockfile generated by Bundler 1.x:

GEM (optional but very common)

These are dependencies sourced from a Rubygems server. That may be the main Rubygems index, at Rubygems.org, or it may be a custom index, such as those available from Gemfury and others. Within this section you'll see:

remote: one or more lines specifying the location of the Rubygems index(es)
specs: a list of dependencies, with their version number, and the constraints on any subdependencies

GIT (optional)

These are dependencies sourced from a given git remote. You'll see a different one of these sections for each git remote, and within each section you'll see:

remote: the git remote. E.g., [email protected]:rails/rails
revision: the commit reference the Gemfile.lock is locked to
tag: (optional) the tag specified in the Gemfile
specs: the git dependency found at this remote, with its version number, and the constraints on any subdependencies

PATH (optional)

These are dependencies sourced from a given path, provided in the Gemfile. You'll see a different one of these sections for each path dependency, and within each section you'll see:

remote: the path. E.g., plugins/vendored-dependency
specs: the git dependency found at this remote, with its version number, and the constraints on any subdependencies

PLATFORMS

The Ruby platform the Gemfile.lock was generated against. If any dependencies in the Gemfile specify a platform then they will only be included in the Gemfile.lock when the lockfile is generated on that platform (e.g., through an install).

DEPENDENCIES

A list of the dependencies which are specified in the Gemfile, along with the version constraint specified there.

Dependencies specified with a source other than the main Rubygems index (e.g., git dependencies, path-based, dependencies) have a ! which means they are "pinned" to that source² (although one must sometimes look in the Gemfile to determine in).

RUBY VERSION (optional)

The Ruby version specified in the Gemfile, when this Gemfile.lock was created. If a Ruby version is specified in a .ruby_version file instead this section will not be present (as Bundler will consider the Gemfile / Gemfile.lock agnostic to the installer's Ruby version).

BUNDLED WITH (Bundler >= v1.10.x)

The version of Bundler used to create the Gemfile.lock. Used to remind installers to update their version of Bundler, if it is older than the version that created the file.

PLUGIN SOURCE (optional and very rare)

In theory, a Gemfile can specify Bundler plugins, as well as gems³, which would then be listed here. In practice, I'm not aware of any available plugins, as of July 2017. This part of Bundler is still under active development!

https://dependabot.com
https://github.com/bundler/bundler/issues/4631
http://andre.arko.net/2012/07/23/towards-a-bundler-plugin-system/

Solution 3:

in regards to the exclamation mark I just found out it's on gems fetched via :git, e.g.

gem "foo", :git => "[email protected]:company/foo.git"

Solution 4:

Bundler is a Gem manager which provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed.

Gemfile and Gemfile.lock are primary products given by Bundler gem (Bundler itself is a gem).

Gemfile contains your project dependency on gem(s), that you manually mention with version(s) specified, but those gem(s) inturn depends on other gem(s) which is resolved by bundler automatically.

Gemfile.lock contain complete snapshot of all the gem(s) in Gemfile along with there associated dependency.

When you first call bundle install, it will create this Gemfile.lock and uses this file in all subsequent calls to bundle install, which ensures that you have all the dependencies installed and will skip dependency installation.

Same happens when you share your code with different machines

You share your Gemfile.lock along with Gemfile, when you run bundle install on other machine it will refer to your Gemfile.lock and skip dependency resolution step, instead it will install all of the same dependent gem(s) that you used on the original machine, which maintains consistency across multiple machines

Why do we need to maintain consistency along multiple machines ?

Running different versions on different machines could lead to broken code
Suppose, your app used the version 1.5.3 and it works 14 months ago
without any problems, and you try to install on different machine
without Gemfile.lock now you get the version 1.5.8. Maybe it's broken with the latest version of some gem(s) and your application will
fail. Maintaining consistency is of utmost importance (preferred
practice).

It is also possible to update gem(s) in Gemfile.lock by using bundle update.

This is based on the concept of conservative updating

Solution 5:

It looks to me like PATH lists the first-generation dependencies directly from your gemspec, whereas GEM lists second-generation dependencies (i.e. what your dependencies depend on) and those from your Gemfile. PATH::remote is . because it relied on a local gemspec in the current directory to find out what belongs in PATH::spec, whereas GEM::remote is rubygems.org, since that's where it had to go to find out what belongs in GEM::spec.

In a Rails plugin, you'll see a PATH section, but not in a Rails app. Since the app doesn't have a gemspec file, there would be nothing to put in PATH.

As for DEPENDENCIES, gembundler.com states:

Runtime dependencies in your gemspec are treated like base dependencies, 
and development dependencies are added by default to the group, :development

The Gemfile generated by rails plugin new my_plugin says something similar:

# Bundler will treat runtime dependencies like base dependencies, and
# development dependencies will be added by default to the :development group.

What this means is that the difference between

s.add_development_dependency "july" # (1)

and

s.add_dependency "july" # (2)

is that (1) will only include "july" in Gemfile.lock (and therefore in the application) in a development environment. So when you run bundle install, you'll see "july" not only under PATH but also under DEPENDENCIES, but only in development. In production, it won't be there at all. However, when you use (2), you'll see "july" only in PATH, not in DEPENDENCIES, but it will show up when you bundle install from a production environment (i.e. in some other gem that includes yours as a dependency), not only development.

These are just my observations and I can't fully explain why any of this is the way it is but I welcome further comments.