How to rebuild a project

#1

The Servo project hasn’t been built in 8 months and is very out of date. Getting it up and working requires a not-insignificant amount of work, including uninstalling Gstreamer and rebuilding it from source. Also the VM is just barely large enough to work on, you can check out and build a branch but there’s not enough space to have two branches built at once.

I’d like to rebuild the Servo docker image so that its up to date with the version in Github and compiles without struggle. I don’t know how to do that.

I found this post detailing how to add a docker file. Is that the way to do this? Servo’s docker file is only 2 months old in Github so why is the latest build 8 months old?

Thanks for any help you can point me to or provide

0 Likes

#2

Hi @robert-snakard, and a warm welcome into our community!

Thank you for reaching out, and I’m very sorry that the image is so old. Our images are automatically built by CircleCI and pushed to Docker Hub, every time we push changes to our Dockerfiles (or when we trigger rebuilds manually on CircleCI).

But sometimes some project images won’t get updated, e.g. when there are build problems with their Dockerfile. To illustrate, here is the last CircleCI build of our core images:

As you can see, the builds of both Fennec and Servo failed. And unfortunately, Servo’s has been broken for quite a while (I’ve tried fixing it a few times, but didn’t succeed). For this latest attempt, CircleCI shows this in the logs:

   Compiling gfx v0.0.1 (/home/user/servo/components/gfx)
   Compiling script_layout_interface v0.0.1 (/home/user/servo/components/script_layout_interface)
   Compiling layout_traits v0.0.1 (/home/user/servo/components/layout_traits)
   Compiling constellation v0.0.1 (/home/user/servo/components/constellation)
   Compiling layout v0.0.1 (/home/user/servo/components/layout)
   Compiling layout_thread v0.0.1 (/home/user/servo/components/layout_thread)
   Compiling libservo v0.0.1 (/home/user/servo/components/servo)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/user/servo/target/debug/deps/servo-a5b4289cadae1ef8.10nquuoxz4qij3mi.rcgu.o" "/home/user/servo/target/debug/deps/servo-a5b4289cadae1ef8.115ga2g0xtxw8v3n.rcgu.o" [...] "/home/user/servo/target/debug/deps/servo-a5b4289cadae1ef8.v05edax6qulk6nw.rcgu.o" "-o" "/home/user/servo/target/debug/deps/servo-a5b4289cadae1ef8" "/home/user/servo/target/debug/deps/servo-a5b4289cadae1ef8.587qravk160790bw.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/user/servo/target/debug/deps" "-L" "/home/user/servo/target/debug/build/backtrace-sys-a0e688000e43f0d3/out" "-L" "/home/user/servo/target/debug/build/libloading-4f26300d41035994/out" [...] "-L" "/home/user/servo/target/debug/build/heartbeats-simple-sys-1d4dee735e6dc392/out/lib" "-L" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/user/servo/target/debug/deps/libservo-55e7ae5495da6218.rlib" "/home/user/servo/target/debug/deps/libwebdriver_server-2e3187f6448eace2.rlib" [...] "/home/user/servo/target/debug/deps/libcfg_if-c30539bf8fed843f.rlib" "-Wl,--start-group" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-e3c09f1b902a42ef.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-c3309e78ea62d0b4.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-ecbb5516b7d95b76.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-06af39d9fce1c660.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-e6fd95c14158108d.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-62d81db5f5fc3381.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-4bd5b0cfdaf3aa86.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-e087cda567df947c.rlib" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-301d7b645a1ea5e6.rlib" "-Wl,--end-group" "/home/user/.rustup/toolchains/nightly-2019-01-28-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-b26c204ff0aa7a0b.rlib" "-Wl,-Bdynamic" "-lstdc++" "-lgstwebrtc-1.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstsdp-1.0" "-lglib-2.0" "-lgstplayer-1.0" "-lgstvideo-1.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstvideo-1.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstaudio-1.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstapp-1.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstbase-1.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgstreamer-1.0" "-lgobject-2.0" "-lglib-2.0" "-lgobject-2.0" "-lglib-2.0" "-lglib-2.0" "-lssl" "-lcrypto" "-lstdc++" "-lpthread" "-lxcb" "-lxcb" "-lxcb" "-lxcb" "-lGL" "-lskia" "-lGL" "-lstdc++" "-lfreetype" "-lbz2" "-lfontconfig" "-lexpat" "-lX11" "-lazure" "-lstdc++" "-lbz2" "-lGL" "-lfontconfig" "-lfreetype" "-lexpat" "-lX11" "-ldbus-1" "-lstdc++" "-lstdc++" "-lz" "-ldl" "-ldl" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-fuse-ld=gold"
  = note: collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
          compilation terminated.
          /home/user/servo/target/debug/deps/libservo_media_gstreamer-36a61167f1844dd5.rlib(servo_media_gstreamer-36a61167f1844dd5.servo_media_gstreamer.d65dh0b8-cgu.7.rcgu.o):servo_media_gstreamer.d65dh0b8-cgu.7:function gstreamer::promise::Promise::new_with_change_func::h195f7ae4babca27a: error: undefined reference to 'gst_promise_new_with_change_func'
          /home/user/servo/target/debug/deps/libservo_media_gstreamer-36a61167f1844dd5.rlib(servo_media_gstreamer-36a61167f1844dd5.servo_media_gstreamer.d65dh0b8-cgu.7.rcgu.o):servo_media_gstreamer.d65dh0b8-cgu.7:function gstreamer::promise::Promise::new_with_change_func::hcbfc777c06a158c9: error: undefined reference to 'gst_promise_new_with_change_func'
          /home/user/servo/target/debug/deps/libservo_media_gstreamer-36a61167f1844dd5.rlib(servo_media_gstreamer-36a61167f1844dd5.servo_media_gstreamer.d65dh0b8-cgu.7.rcgu.o):servo_media_gstreamer.d65dh0b8-cgu.7:function gstreamer::promise::Promise::new_with_change_func::hdd4b77a54531684a: error: undefined reference to 'gst_promise_new_with_change_func'
          /home/user/servo/target/debug/deps/libgstreamer_player-f5fc3571232f2372.rlib(gstreamer_player-f5fc3571232f2372.gstreamer_player.mws44ddg-cgu.11.rcgu.o):gstreamer_player.mws44ddg-cgu.11:function gstreamer_player::config::PlayerConfig::set_position_update_interval::h413c3df221c3c11e: error: undefined reference to 'gst_player_config_set_position_update_interval'
          /home/user/servo/target/debug/deps/libgstreamer_player-f5fc3571232f2372.rlib(gstreamer_player-f5fc3571232f2372.gstreamer_player.mws44ddg-cgu.8.rcgu.o):gstreamer_player.mws44ddg-cgu.8:function gstreamer_player::auto::player_media_info::PlayerMediaInfo::get_number_of_video_streams::hc7a070e7c79b5336: error: undefined reference to 'gst_player_media_info_get_number_of_video_streams'
          /home/user/servo/target/debug/deps/libgstreamer_player-f5fc3571232f2372.rlib(gstreamer_player-f5fc3571232f2372.gstreamer_player.mws44ddg-cgu.8.rcgu.o):gstreamer_player.mws44ddg-cgu.8:function gstreamer_player::auto::player_media_info::PlayerMediaInfo::get_video_streams::h410cb1501e2cda26: error: undefined reference to 'gst_player_media_info_get_video_streams'
          /home/user/servo/target/debug/deps/libgstreamer_player-f5fc3571232f2372.rlib(gstreamer_player-f5fc3571232f2372.gstreamer_player.mws44ddg-cgu.8.rcgu.o):gstreamer_player.mws44ddg-cgu.8:function gstreamer_player::auto::player_media_info::PlayerMediaInfo::is_live::h84caa021d9218783: error: undefined reference to 'gst_player_media_info_is_live'
          

error: aborting due to previous error

error: Could not compile `servo`.

To learn more, run the command again with --verbose.
[Warning] Could not generate notification! Optional Python module 'dbus' is not installed.
Build FAILED in 0:59:08
The command '/bin/sh -c ./mach build -d' returned a non-zero code: 101
Exited with code 101

It seems that ld failed with a Segmentation Fault while trying to link Servo. This means that it was almost finished, which is a good sign.

However, I’m not sure how to fix or work around that segfault. Maybe we should reach out to more knowledgeable people like Josh Matthews or Simon Sapin.

0 Likes

#3

On IRC, ferjm says that this problem looks like:

0 Likes

#4

I’m trying something: https://github.com/JanitorTechnology/dockerfiles/pull/225

Let’s see if this makes the Servo build happier.

0 Likes

#5

Hey Jan, the linker errors are due to an outdated version of gstreamer. You can see the comments on the Servo issues here and here if you’d like. I believe we either need to update the Ubuntu version to one with a newer version of gstreamer or ship an image that has a custom-built version of gstreamer.

1 Like

#6

Hi Robert, thanks for the explanation!

In the Servo Dockerfile, we already use ./mach bootstrap-gstreamer, which should in theory install a suitable version of gstreamer.

However, we also install some other libgstreamer packages, which might cause conflicts (I’m not sure).

I’ve tried removing the extra libgstreamer packages in this pull request, and the Servo build now fails with a different error:

   Compiling new_debug_unreachable v1.0.1
   Compiling lock_api v0.1.5
   Compiling tokio-current-thread v0.1.1
   Compiling tokio-timer v0.2.6
error: failed to run custom build command for `unwind-sys v0.1.1`
process didn't exit successfully: `/home/user/servo/target/debug/build/unwind-sys-d42f3538b7e8a89b/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Failure { command: "\"pkg-config\" \"--libs\" \"--cflags\" \"libunwind-generic\"", output: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "Package libunwind-generic was not found in the pkg-config search path.\nPerhaps you should add the directory containing `libunwind-generic.pc\'\nto the PKG_CONFIG_PATH environment variable\nNo package \'libunwind-generic\' found\n" } }', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

warning: build failed, waiting for other jobs to finish...
error: build failed
[Warning] Could not generate notification! Optional Python module 'dbus' is not installed.
Build FAILED in 0:14:45
The command '/bin/sh -c ./mach build -d' returned a non-zero code: 101
Exited with code 101
0 Likes

#7

Ah, fixed that with a sudo apt-get install libunwind-devfrom this comment

1 Like

#8

Ah, I see @notriddle already did that here. Thanks! I’m slowly getting the hang of these Dockerfiles now!

1 Like

#9

@notriddle got the Servo image to build again! Many thanks. :smile:

CircleCI is now building all images again, and once a new Servo image is deployed, I’ll make Janitor pull it into production (to replace the 8-months old image!)

Thanks all so much. It makes me so happy that Servo builds again!

0 Likes

#10

@robert-snakard fresh new Servo image on Janitor! :tada:

Please try it when you have time (hopefully it will work well for you).

0 Likes

#11

Works great! and I can compile two slightly different branches at once now! Thanks Jan!

0 Likes