Listing SRC_URI of all packages/files needed to build Yocto image

I would like to list all files that bitbake will fetch when I bake an image.

Currently, I am able to get the SRC_URI of all files needed to bake a Yocto image by executing bitbake core-image-minimal -c fetchall and then parse log files.

Is there is a simpler way to get the same result without the need to download files ?

I am not sure bitbake supports such feature. Ideally, I am looking for a command that prints out the package name and lists all files with corresponding URL

> bitbake core-image-minimal -c fetchall --print-only

Solution 1:

Generally bitbake doesn't provides such functionality.

But I was able to create a simple solution witch creating simple .bbclass file which is inherited in all recipes, by adding it into local.conf file, please see my steps in order to archive that:

Steps:

1. let's create a class print-src.bbclass file used to get and print SRC_URI variable (remember to store this class file in layer which is available in conf/bblayers.conf):

   $ cat print-src.bbclass
   
   python do_print_src () {
       # should probably be indented
       srcuri = d.getVar('SRC_URI', True).split()
       bb.warn("SRC_URI look like: %s" % srcuri)
   }
   
   addtask do_print_src before do_fetch
   

2. Add INHERIT += "print-src" into Your conf/local.conf file

Edit: it is important to use bitbake --runonly option, that allows to run specific task of the taskgraph for the specified target (with --runonly option do_print_src needs to be used as print_src),

Edit: Please note that --runall=RUNALL and --runonly=RUNONLY was introduced with Yocto Sumo release 2.5,

$ bitbake core-image-minimal --runonly print_src
Loaded 1236 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.37.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal-4.8"
TARGET_SYS           = "i586-poky-linux"
MACHINE              = "qemux86"
DISTRO               = "poky"
DISTRO_VERSION       = "2.5"
TUNE_FEATURES        = "m32 i586"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:13cc30cd7de4841990b600e83e1249c81a5171dd"

Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
WARNING: ptest-runner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src: SRC_URI look like: ['git://git.yoctoproject.org/ptest-runner2']
WARNING: grep-3.1-r0 do_print_src: SRC_URI look like: ['http://ftp.gnu.org/gnu/grep/grep-3.1.tar.xz', 'file://0001-Unset-need_charset_alias-when-building-for-musl.patch']
...
... 
NOTE: Tasks Summary: Attempted 201 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There were 202 WARNING messages shown.

Please see sample warning output log line:

WARNING: ptest-runner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src: SRC_URI look like: ['git://git.yoctoproject.org/ptest-runner2'].

Solution 2:

I patched poky to create *.src files in downloads that contains the effective fetch URL of the package.

 bitbake/lib/bb/fetch2/__init__.py | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index b853da30bd..03e84e0919 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -1257,16 +1257,16 @@ class FetchData(object):

         # Note: .done and .lock files should always be in DL_DIR whereas localpath may not be.
         if self.localpath and self.localpath.startswith(dldir):
-            basepath = self.localpath
+            self.basepath = self.localpath
         elif self.localpath:
-            basepath = dldir + os.sep + os.path.basename(self.localpath)
+            self.basepath = dldir + os.sep + os.path.basename(self.localpath)
         elif self.basepath or self.basename:
-            basepath = dldir + os.sep + (self.basepath or self.basename)
+            self.basepath = dldir + os.sep + (self.basepath or self.basename)
         else:
              bb.fatal("Can't determine lock path for url %s" % url)

-        self.donestamp = basepath + '.done'
-        self.lockfile = basepath + '.lock'
+        self.donestamp = self.basepath + '.done'
+        self.lockfile = self.basepath + '.lock'

     def setup_revisions(self, d):
         self.revisions = {}
@@ -1607,6 +1607,15 @@ class Fetch(object):
             m = ud.method
             localpath = ""

+            p = "%s.src"%ud.basepath
+            d = os.path.dirname(p)
+            if d != '':
+                bb.utils.mkdirhier(d)
+            with open(p, 'wb') as f:
+                data = "%s" % ud.url
+                f.write(bytes(data, 'ASCII'))
+            return True
+
             if ud.lockfile:
                 lf = bb.utils.lockfile(ud.lockfile)

Running bitbake core-image-minimal -c fetchall results:

$> find downloads/ -name '*.src' | head -n 5
downloads/lzop-1.03.tar.gz.src
downloads/libtheora-1.1.1.tar.bz2.src
downloads/mpfr-3.1.5.tar.xz.src
downloads/makedevs.c.src
downloads/expat-2.2.0.tar.bz2.src

This is not an optimal solution but I hope that such feature gets its way to mainline stream.