diff options
Diffstat (limited to 'dev/ci')
| -rw-r--r-- | dev/ci/README.md | 38 | ||||
| -rwxr-xr-x | dev/ci/ci-fiat-crypto.sh | 5 | ||||
| -rwxr-xr-x | dev/ci/ci-math-classes.sh | 2 | ||||
| -rwxr-xr-x | dev/ci/ci-pidetop.sh | 15 | ||||
| -rw-r--r-- | dev/ci/docker/README.md | 41 | ||||
| -rw-r--r-- | dev/ci/docker/bionic_coq/Dockerfile | 3 | ||||
| -rwxr-xr-x | dev/ci/docker/bionic_coq/hooks/post_push | 8 | ||||
| -rw-r--r-- | dev/ci/user-overlays/06859-ejgallego-stm+top.sh | 6 | ||||
| -rw-r--r-- | dev/ci/user-overlays/07196-ejgallego-tactics+push_fix_naming_out.sh | 21 | ||||
| -rw-r--r-- | dev/ci/user-overlays/README.md | 17 |
10 files changed, 95 insertions, 61 deletions
diff --git a/dev/ci/README.md b/dev/ci/README.md index dee3d2aff7..697a160ca9 100644 --- a/dev/ci/README.md +++ b/dev/ci/README.md @@ -107,19 +107,32 @@ there are some. You can also run one CI target locally (using `make ci-somedev`). -Whenever your PR breaks tested developments, you should either adapt it -so that it doesn't, or provide a branch fixing these developments (or at -least work with the author of the development / other Coq developers to -prepare these fixes). Then, add an overlay in -[`dev/ci/user-overlays`](/dev/ci/user-overlays) (see the README there) -as part of your PR. +See also [`test-suite/README.md`](/test-suite/README.md) for information about adding new tests to the test-suite. -The process to merge your PR is then to submit PRs to the external -development repositories, merge the latter first (if the fixes are -backward-compatible), and merge the PR on Coq then. +### Breaking changes -See also [`test-suite/README.md`](/test-suite/README.md) for information about adding new tests to the test-suite. +When your PR breaks an external project we test in our CI, you must prepare a +patch (or ask someone to prepare a patch) to fix the project: + +1. Fork the external project, create a new branch, push a commit adapting + the project to your changes. +2. Test your pull request with your adapted version of the external project by + adding an overlay file to your pull request (cf. + [`dev/ci/user-overlays/README.md`](/dev/ci/user-overlays/README.md)). +3. Fixes to external libraries (pure Coq projects) *must* be backward + compatible (i.e. they should also work with the development version of Coq, + and the latest stable version). This will allow you to open a PR on the + external project repository to have your changes merged *before* your PR on + Coq can be integrated. + On the other hand, patches to plugins (projects linking to the Coq ML API) + can very rarely be made backward compatible and plugins we test will + generally have a dedicated branch per Coq version. + You can still open a pull request but the merging will be requested by the + developer who merges the PR on Coq. There are plans to improve this, cf. + [#6724](https://github.com/coq/coq/issues/6724). + +Moreover your PR must absolutely update the [`CHANGES`](/CHANGES) file. Advanced GitLab CI information ------------------------------ @@ -141,7 +154,6 @@ no OCaml warnings build Coq in parallel with other tests. ### GitLab and Windows - If your repository has access to runners tagged `windows`, setting the secret variable `WINDOWS` to `enabled` will add jobs building Windows versions of Coq (32bit and 64bit). @@ -155,6 +167,10 @@ System and opam packages are installed in a Docker image. The image is automatically built and uploaded to your GitLab registry, and is loaded by subsequent jobs. +**IMPORTANT**: When updating Coq's CI docker image, you must modify +the `CACHEKEY` variable in `.gitlab-ci.yml`, `.circleci/config.yml`, +and `Dockerfile`. + The Docker building job reuses the uploaded image if it is available, but if you wish to save more time you can skip the job by setting `SKIP_DOCKER` to `true`. diff --git a/dev/ci/ci-fiat-crypto.sh b/dev/ci/ci-fiat-crypto.sh index 5d96c24991..feabf72d48 100755 --- a/dev/ci/ci-fiat-crypto.sh +++ b/dev/ci/ci-fiat-crypto.sh @@ -9,5 +9,6 @@ git_checkout "${fiat_crypto_CI_BRANCH}" "${fiat_crypto_CI_GITURL}" "${fiat_crypt ( cd "${fiat_crypto_CI_DIR}" && git submodule update --init --recursive ) -fiat_crypto_CI_TARGETS="print-nobigmem nobigmem nonautogenerated-specific nonautogenerated-specific-display" -( cd "${fiat_crypto_CI_DIR}" && make ${fiat_crypto_CI_TARGETS} ) +fiat_crypto_CI_TARGETS1="printlite lite lite-display" +fiat_crypto_CI_TARGETS2="print-nobigmem nobigmem nonautogenerated-specific nonautogenerated-specific-display" +( cd "${fiat_crypto_CI_DIR}" && make ${fiat_crypto_CI_TARGETS1} && make ${fiat_crypto_CI_TARGETS2} ) diff --git a/dev/ci/ci-math-classes.sh b/dev/ci/ci-math-classes.sh index 4fc06e8956..6a064b2971 100755 --- a/dev/ci/ci-math-classes.sh +++ b/dev/ci/ci-math-classes.sh @@ -7,4 +7,4 @@ math_classes_CI_DIR="${CI_BUILD_DIR}/math-classes" git_checkout "${math_classes_CI_BRANCH}" "${math_classes_CI_GITURL}" "${math_classes_CI_DIR}" -( cd "${math_classes_CI_DIR}" && make && make install ) +( cd "${math_classes_CI_DIR}" && ./configure.sh && make && make install ) diff --git a/dev/ci/ci-pidetop.sh b/dev/ci/ci-pidetop.sh index d04b9637c0..2ac4d21671 100755 --- a/dev/ci/ci-pidetop.sh +++ b/dev/ci/ci-pidetop.sh @@ -8,6 +8,17 @@ pidetop_CI_DIR="${CI_BUILD_DIR}/pidetop" git_checkout "${pidetop_CI_BRANCH}" "${pidetop_CI_GITURL}" "${pidetop_CI_DIR}" -( cd "${pidetop_CI_DIR}" && coq_makefile -f Make -o Makefile.top && make -f Makefile.top "-j${NJOBS}" && make install-toploop -f Makefile.top ) +# Travis / Gitlab have different filesystem layout due to use of +# `-local`. We need to improve this divergence but if we use Dune this +# "local" oddity goes away automatically so not bothering... +if [ -d "$COQBIN/../lib/coq" ]; then + COQOCAMLLIB="$COQBIN/../lib/" + COQLIB="$COQBIN/../lib/coq/" +else + COQOCAMLLIB="$COQBIN/../" + COQLIB="$COQBIN/../" +fi -echo -en '4\nexit' | coqtop -main-channel stdfds -toploop pidetop +( cd "${pidetop_CI_DIR}" && OCAMLPATH="$COQOCAMLLIB" jbuilder build @install ) + +echo -en '4\nexit' | "$pidetop_CI_DIR/_build/install/default/bin/pidetop" -coqlib "$COQLIB" -main-channel stdfds diff --git a/dev/ci/docker/README.md b/dev/ci/docker/README.md index 8e677f6f2b..919e2a735f 100644 --- a/dev/ci/docker/README.md +++ b/dev/ci/docker/README.md @@ -1,37 +1,13 @@ ## Overall Docker Setup for Coq's CI. This directory provides Docker images to be used by Coq's CI. The -images do support Docker autobuild on `hub.docker.com` +images do support Docker autobuild on `hub.docker.com` and Gitlab's +private registry. -Autobuild is the preferred build method [see below]; if you are a -member of the `coqci` organization, the automated build will push the -image to the `coqci/name:$VERSION` tag using a Docker hub hook. - -## Updating the Image and Syncronization with CI files - -Unfortunately, at this point some manual synchronization is needed -between the `Dockerfile` and `.gitlab-ci.yml`. In particular, the -checklist is: - -- check the name and version of the image setup in `hooks/post_push` - have to match. -- check `COMPILER` variables do match. - -Once you are sure the variables are right, then proceed to trigger an -autobuild or do a manual build, et voilĂ ! - -## Docker Autobuilding. - -We provide basic support for auto-building, see: - -https://docs.docker.com/docker-cloud/builds/advanced/ - -If you are member of the `coqci` Docker organization, trigger an -autobuild in your account and the image will be pushed to it as we -set the proper tag in `hooks/post_push`. - -We still need to figure out how properly setup a more automated, -organization-wide auto-building process. +Gitlab CI will build and tag a Docker by default for every job if the +`SKIP_DOCKER` variable is not set to `false`. In Coq's CI, this +variable is usually set to `false` indeed to avoid booting a useless +job. ## Manual Building @@ -47,10 +23,6 @@ To upload/push to your hub: + `docker tag base:$VERSION $USER/base:$VERSION` + `docker push $USER/base:$VERSION` -Now your docker image is ready to be used. To upload to `coqci`: -- `docker tag base:$VERSION coqci/base:$VERSION` -- `docker push coqci/base:$VERSION` - ## Debugging / Misc To open a shell inside an image do `docker run -ti --entrypoint /bin/bash <imageID>` @@ -62,4 +34,3 @@ end. ## Possible Improvements: - Use ARG for customizing versions, centralize variable setup; -- Learn more about Docker registry for GITLAB https://gitlab.com/coq/coq/container_registry . diff --git a/dev/ci/docker/bionic_coq/Dockerfile b/dev/ci/docker/bionic_coq/Dockerfile index 689d203a16..a1178ee2a0 100644 --- a/dev/ci/docker/bionic_coq/Dockerfile +++ b/dev/ci/docker/bionic_coq/Dockerfile @@ -1,3 +1,6 @@ +# CACHEKEY: "bionic_coq-V2018-05-07-V2" +# ^^ Update when modifying this file. + FROM ubuntu:bionic LABEL maintainer="e@x80.org" diff --git a/dev/ci/docker/bionic_coq/hooks/post_push b/dev/ci/docker/bionic_coq/hooks/post_push deleted file mode 100755 index 307680aa51..0000000000 --- a/dev/ci/docker/bionic_coq/hooks/post_push +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env bash - -COQCI_VERSION=V2018-05-07-V2 -docker tag $IMAGE_NAME $DOCKER_REPO:$COQCI_VERSION -docker push $DOCKER_REPO:$COQCI_VERSION - -docker tag $IMAGE_NAME coqci/base:$COQCI_VERSION -docker push coqci/base:$COQCI_VERSION diff --git a/dev/ci/user-overlays/06859-ejgallego-stm+top.sh b/dev/ci/user-overlays/06859-ejgallego-stm+top.sh new file mode 100644 index 0000000000..ca0076b468 --- /dev/null +++ b/dev/ci/user-overlays/06859-ejgallego-stm+top.sh @@ -0,0 +1,6 @@ +#!/bin/sh + +if [ "$CI_PULL_REQUEST" = "6859" ] || [ "$CI_BRANCH" = "stm+top" ] || [ "$CI_BRANCH" = "pr-6859" ]; then + pidetop_CI_BRANCH=stm+top + pidetop_CI_GITURL=https://bitbucket.org/ejgallego/pidetop.git +fi diff --git a/dev/ci/user-overlays/07196-ejgallego-tactics+push_fix_naming_out.sh b/dev/ci/user-overlays/07196-ejgallego-tactics+push_fix_naming_out.sh new file mode 100644 index 0000000000..ea9cd8ee07 --- /dev/null +++ b/dev/ci/user-overlays/07196-ejgallego-tactics+push_fix_naming_out.sh @@ -0,0 +1,21 @@ +if [ "$CI_PULL_REQUEST" = "7196" ] || [ "$CI_BRANCH" = "tactics+push_fix_naming_out" ] || [ "$CI_BRANCH" = "pr-7196" ]; then + + # Needed overlays: https://gitlab.com/coq/coq/pipelines/21244550 + # + # equations + # ltac2 + + # The below developments should instead use a backwards compatible fix. + # + # color + # iris-lambda-rust + # math-classes + # formal-topology + + ltac2_CI_BRANCH=tactics+push_fix_naming_out + ltac2_CI_GITURL=https://github.com/ejgallego/ltac2 + + Equations_CI_BRANCH=tactics+push_fix_naming_out + Equations_CI_GITURL=https://github.com/ejgallego/Coq-Equations + +fi diff --git a/dev/ci/user-overlays/README.md b/dev/ci/user-overlays/README.md index a7474e3248..aec2dfe0a6 100644 --- a/dev/ci/user-overlays/README.md +++ b/dev/ci/user-overlays/README.md @@ -1,8 +1,21 @@ # Add overlays for your pull requests in this directory -An overlay is a file containing very simple logic to test whether we are currently building a specific pull request or git branch (useful so that overlays work on your own fork) and which changes some of the variables whose default can be found in [`ci-basic-overlay.sh`](/dev/ci/ci-basic-overlay.sh). +When your pull request breaks an external project we test in our CI and you +have prepared a branch with the fix, you can add an "overlay" to your pull +request to test it with the adapted version of the external project. -The name of your overlay file should be of the form `five_digit_PR_number-GitHub_handle-branch_name.sh`. +An overlay is a file which defines where to look for the patched version so that +testing is possible. It redefines some variables from +[`ci-basic-overlay.sh`](/dev/ci/ci-basic-overlay.sh): +give the name of your branch using a `_CI_BRANCH` variable and the location of +your fork using a `_CI_GITURL` variable. + +Moreover, the file contains very simple logic to test the pull request number +or branch name and apply it only in this case. + +The name of your overlay file should start with a five-digit pull request +number, followed by a dash, anything (for instance your GitHub nickname +and the branch name), then a `.sh` extension (`[0-9]{5}-[a-zA-Z0-9-_]+.sh`). Example: `00669-maximedenes-ssr-merge.sh` containing |
