Compare commits

...

25 commits

Author SHA1 Message Date
cf383a9995 Renovate: Update dependency pre-commit to v3.8.0
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-29 07:03:18 +00:00
d44a4ef685 Renovate: Update woodpeckerci/plugin-git Docker digest to a878e6f
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-26 09:02:59 +00:00
980b4510ed Renovate: migrate config renovate.json
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-25 14:23:03 +00:00
f39fcf4d82 Renovate: Update all dependencies
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-24 13:02:38 +00:00
46bff875b5 Renovate: Update woodpeckerci/plugin-git Docker digest to 1b6f184
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-15 07:02:52 +00:00
933ddf30d0 Renovate: Update python:3.12.4-alpine3.20 Docker digest to 0bd77ae
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-11 07:16:02 +00:00
135f60659a Renovate: Update woodpeckerci/plugin-docker-buildx Docker tag to v4.1.0
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-10 07:16:26 +00:00
424a53ef02 Renovate: Update pre-commit hook gitleaks/gitleaks to v8.18.4
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-03 16:56:32 +00:00
24ee40e4be Renovate: Update python:3.12.4-alpine3.20 Docker digest to b7662fc
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/manual/build Pipeline was successful
2024-07-03 16:31:05 +00:00
161a73e8e9 Renovate: Update all dependencies
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-07-03 10:02:20 +00:00
73f802fab0 Renovate: Update python:3.12.4-alpine3.20 Docker digest to ff870bf
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-06-27 07:02:22 +00:00
9354a361a3
Add package version script
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-06-25 16:18:57 +02:00
e43b05e0fd Renovate: Pin dependencies
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2024-06-25 07:02:53 +00:00
64c3f1fa9c
Add tests for pre-commit container
Some checks failed
ci/woodpecker/push/build Pipeline failed
ci/woodpecker/push/test Pipeline failed
Run pre-commit on the freshly built container against salt and rc2matrix
2024-06-20 23:44:19 +02:00
7b022827a1
Remove dependencies to let pre-commit install them
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-06-20 16:26:50 +02:00
4e449c56a7
Add README
Some checks failed
ci/woodpecker/push/woodpecker Pipeline was successful
ci/woodpecker/manual/woodpecker Pipeline failed
2024-06-20 12:00:53 +02:00
2fd071bb78
Update gitleaks to use native hook
All checks were successful
ci/woodpecker/manual/woodpecker Pipeline was successful
2024-06-20 11:37:19 +02:00
90590626b8
Run pre-commit to warm up cache
Some checks failed
ci/woodpecker/push/woodpecker Pipeline was successful
ci/woodpecker/manual/woodpecker Pipeline failed
2024-06-19 17:18:32 +02:00
94a4817e70 Renovate: Update python Docker tag to v3.12.4
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-06-19 11:55:58 +00:00
53d8640163
Copy gitleaks from container
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-06-19 13:36:12 +02:00
fbc13e5630
Install gitleaks binary from github releases
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-06-19 10:11:01 +02:00
dd09aab113 Renovate: Update python:3.12.3-alpine3.20 Docker digest to 32385e6
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-06-06 07:04:37 +00:00
a1a0cfc40d
Upgrade to alpine 3.20
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-06-04 11:52:58 +02:00
ef85aab842
Add ruby-devel package
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2024-05-17 15:59:30 +02:00
4afb2cec82 Renovate: Update all dependencies
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
2024-05-16 07:04:18 +00:00
13 changed files with 282 additions and 59 deletions

View file

@ -1,2 +1,3 @@
# Ignore everything
*
!.pre-commit-config.yaml

16
.editorconfig Normal file
View file

@ -0,0 +1,16 @@
root = true
[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
# Non-standard
quote_type = single
[*.{diff,patch}]
indent_style = unset
indent_size = unset

2
.markdown-style.rb Normal file
View file

@ -0,0 +1,2 @@
all # Import all rules
exclude_rule "MD013" # Ignore Line length

2
.mdlrc Normal file
View file

@ -0,0 +1,2 @@
style "#{File.dirname(__FILE__)}/.markdown-style.rb"
git_recurse true

55
.pre-commit-config.yaml Normal file
View file

@ -0,0 +1,55 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-json
- id: check-merge-conflict
- id: check-symlinks
- id: check-xml
- id: check-yaml
- id: double-quote-string-fixer
- id: end-of-file-fixer
- id: fix-byte-order-marker
- id: mixed-line-ending
- id: no-commit-to-branch
- id: requirements-txt-fixer
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- repo: https://github.com/warpnet/salt-lint
rev: v0.9.2
hooks:
- id: salt-lint
- repo: https://github.com/markdownlint/markdownlint
rev: v0.13.0
hooks:
- id: markdownlint
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.9.0.5
hooks:
- id: shellcheck
- repo: https://github.com/gitleaks/gitleaks
rev: v8.18.4
hooks:
- id: gitleaks
- repo: local
hooks:
- id: check-ssh-keys
name: check SSH public keys in user pillars
entry: python build/check-ssh-keys.py
language: python
files: ^pillars/users/.+\.sls$
additional_dependencies: ['pyyaml==6.0.1'] # Renovate can't parse it, yet https://github.com/renovatebot/renovate/issues/20780 # TODO
- id: prettier # Copied from https://github.com/pre-commit/mirrors-prettier/ instead of referencing it to not rely on their published Prettier versions
name: Prettier
description: ''
entry: prettier --write --ignore-unknown
language: node
'types': [text]
args: []
require_serial: false
additional_dependencies: ['prettier@3'] # Renovate can't parse this, either. Unspecific to prevent local installs, when global installations are available
minimum_pre_commit_version: '0'

4
.prettierrc.yaml Normal file
View file

@ -0,0 +1,4 @@
semi: false
bracketSpacing: true
trailingComma: es5
proseWrap: preserve

View file

@ -1,32 +0,0 @@
when:
path: "*Dockerfile*"
steps:
build-main:
when:
branch: main
image: woodpeckerci/plugin-docker-buildx:3.2.1@sha256:a4a4e4cfd6ca3a8234d7bc87e771fea0ced1326d174e2620f670331358cbcddb
pull: true
settings:
platforms: linux/amd64
registry: ${CI_FORGE_URL}
username: WoodpeckerCI
password:
from_secret: gitea_token
repo: git.verdigado.com/${CI_REPO,,}
tag: "latest"
build-branch:
when:
branch:
exclude: ["main"]
image: woodpeckerci/plugin-docker-buildx:3.2.1@sha256:a4a4e4cfd6ca3a8234d7bc87e771fea0ced1326d174e2620f670331358cbcddb
pull: true
settings:
platforms: linux/amd64
registry: ${CI_FORGE_URL}
username: WoodpeckerCI
password:
from_secret: gitea_token
repo: git.verdigado.com/${CI_REPO,,}
tag: ${CI_COMMIT_BRANCH}

35
.woodpecker/.build.yaml Normal file
View file

@ -0,0 +1,35 @@
steps:
build main:
when:
- event: push
branch: main
image: woodpeckerci/plugin-docker-buildx:4.2.0@sha256:e3c7a04b5c1c679655a7f8de77721a39492019b4c372bea0e90ec3dd765e750a
pull: true
settings:
platforms: linux/amd64
registry: ${CI_FORGE_URL}
username: WoodpeckerCI
password:
from_secret: gitea_token
repo: git.verdigado.com/${CI_REPO,,}
tags:
- 'latest'
- ${CI_COMMIT_SHA}
build branch:
when:
- event: push
branch:
exclude: ['main']
image: woodpeckerci/plugin-docker-buildx:4.2.0@sha256:e3c7a04b5c1c679655a7f8de77721a39492019b4c372bea0e90ec3dd765e750a
pull: true
settings:
platforms: linux/amd64
registry: ${CI_FORGE_URL}
username: WoodpeckerCI
password:
from_secret: gitea_token
repo: git.verdigado.com/${CI_REPO,,}
tags:
- ${CI_COMMIT_BRANCH}
- ${CI_COMMIT_SHA}

56
.woodpecker/.test.yaml Normal file
View file

@ -0,0 +1,56 @@
skip_clone: true
when:
- event: push
depends_on:
- build
variables:
- &image 'git.verdigado.com/verdigado-images/container-pre-commit:${CI_COMMIT_SHA}'
steps:
await-image:
image: alpine@sha256:0a4eaa0eecf5f8c050e5bba433f58c052be7587ee8af3e8b3910ef9ab5fbe9f5
environment:
IMAGE: *image
commands:
- apk add --update --no-cache img
- 'while !(( img pull $IMAGE 2>&1 | grep -q "Error: failed to unmount" )) ; do echo "Awaiting image $IMAGE..."; sleep 3; done'
- echo 'found.'
clone salt:
image: woodpeckerci/plugin-git@sha256:a878e6f9674d44c0dc43dcb6d8b916507b21176ab44fac70567af96cb80de602
settings:
remote: https://git.verdigado.com/verdigado-Privileged/Salt.git
path: salt
sha: ''
ref: refs/heads/master
branch: master
pre-commit salt:
image: *image
depends_on:
- await-image
- clone salt
environment:
- SKIP=no-commit-to-branch # Ignore "don't commit to protected branch" check
commands:
- cd salt
- pre-commit run --all-files
clone rocketchat2matrix:
image: woodpeckerci/plugin-git@sha256:a878e6f9674d44c0dc43dcb6d8b916507b21176ab44fac70567af96cb80de602
settings:
remote: https://git.verdigado.com/NB-Public/rocketchat2matrix.git
path: rocketchat2matrix
sha: ''
ref: refs/heads/main
branch: master
pre-commit rocketchat2matrix:
image: *image
depends_on:
- await-image
- clone rocketchat2matrix
environment:
- SKIP=no-commit-to-branch # Ignore "don't commit to protected branch" check
commands:
- cd rocketchat2matrix
- pre-commit run --all-files

View file

@ -1,31 +1,33 @@
FROM python:3.12.3-alpine3.19@sha256:ef097620baf1272e38264207003b0982285da3236a20ed829bf6bbf1e85fe3cb
COPY --from=koalaman/shellcheck:v0.10.0@sha256:2097951f02e735b613f4a34de20c40f937a6c8f18ecb170612c88c34517221fb /bin/shellcheck /bin/
FROM python:3.12.4-alpine3.20@sha256:7f15e22f496c65cffbbac5e30e7e98d60f3e3b9cc5ee5d51cf3c55ed604787c8
# renovate: datasource=repology depName=alpine_3_19/build-base versioning=loose
# renovate: datasource=repology depName=alpine_3_20/build-base versioning=loose
ENV BUILD_BASE_VERSION="0.5-r3"
# renovate: datasource=repology depName=alpine_3_19/gcc versioning=loose
ENV GCC_VERSION="13.2.1_git20231014-r0"
# renovate: datasource=repology depName=alpine_3_19/ruby versioning=loose
ENV RUBY_VERSION="3.2.2-r1"
# renovate: datasource=repology depName=alpine_3_19/git versioning=loose
ENV GIT_VERSION="2.43.0-r0"
# renovate: datasource=repology depName=alpine_3_19/openssh-keygen versioning=loose
ENV OPENSSH_KEYGEN_VERSION="9.6_p1-r0"
# renovate: datasource=repology depName=alpine_3_20/gcc versioning=loose
ENV GCC_VERSION="13.2.1_git20240309-r0"
# renovate: datasource=repology depName=alpine_3_20/ruby versioning=loose
ENV RUBY_VERSION="3.3.3-r0"
# renovate: datasource=repology depName=alpine_3_20/git versioning=loose
ENV GIT_VERSION="2.45.2-r0"
# renovate: datasource=repology depName=alpine_3_20/openssh-keygen versioning=loose
ENV OPENSSH_KEYGEN_VERSION="9.7_p1-r4"
# renovate: datasource=pypi depName=pre-commit versioning=pep440
ENV PRE_COMMIT_VERSION="3.7.0"
# renovate: datasource=rubygems depName=mdl versioning=ruby
ENV MDL_VERSION="0.13.0"
ENV PRE_COMMIT_VERSION="3.8.0"
RUN mkdir /data /tmp/pre-commit
COPY .pre-commit-config.yaml /tmp/pre-commit
RUN apk add --update --no-cache \
build-base="${BUILD_BASE_VERSION}" \
gcc="${GCC_VERSION}" \
ruby="${RUBY_VERSION}" \
ruby-dev="${RUBY_VERSION}" \
git="${GIT_VERSION}" \
openssh-keygen="${OPENSSH_KEYGEN_VERSION}" \
&& \
pip install --no-cache-dir pre-commit=="${PRE_COMMIT_VERSION}" && \
gem install --no-document mdl -v "${MDL_VERSION}" && \
mkdir /data && \
git config --global --add safe.directory /data
git config --global --add safe.directory /data && \
cd /tmp/pre-commit && \
git init --initial-branch main && \
pre-commit install --install-hooks && \
rm -rf /tmp/pre-commit
WORKDIR /data

73
README.md Normal file
View file

@ -0,0 +1,73 @@
# verdigado pre-commit container
A container image to include all dependencies (and a warmed up cache) used in our [`pre-commit`](https://pre-commit.com/) hooks/CI steps to speed up execution.
If you see any pre-commit CI jobs installing dependencies:
- Make sure to execute `pre-commit` using this container
- Add the hook to this repo's `.pre-commit-config.yaml`
- Optionally install dependencies in the `Dockerfile` with the versions set up for `Renovate`
## Usage
In your `.woodpecker.yaml`, adapt and add the following block:
```yaml
steps:
check-pre-commit:
image: git.verdigado.com/verdigado-images/container-pre-commit:latest
environment:
- SKIP=no-commit-to-branch # Ignore "don't commit to protected branch" check
commands:
- pre-commit run --all-files
```
If renovate is set up for your repo, it'll add and update the pinned digest/hash of the image.
## Development
Generally you should have `Docker` or something alike installed.
If you need to copy files into the container, don't forget to add exclusions to the general _exclude all_ in `.dockerignore`.
To **update the base image** (like `3.12.4-alpine3.20` to a newer Alpine version), manual work is still required, but supported by a little script. **Renovate might not create a PR for newer image tags.**
1. In the `Dockerfile`, update the Alpine version for the image and the renovate comments (`# renovate: datasource=repology depName=alpine_3_20/gcc versioning=loose`).
```diff
- FROM python:3-alpine3.19@sha256:00c0ffeeacab...
+ FROM python:3-alpine3.20 # You can omit the sha256 digest, the script prints it out
# ...
- # renovate: datasource=repology depName=alpine_3_19/build-base versioning=loose
+ # renovate: datasource=repology depName=alpine_3_20/build-base versioning=loose
ENV BUILD_BASE_VERSION="0.8.15"
# ...
```
1. Now run `./get_pkg_versions.sh`. It pulls the alpine image from the Dockerfile, prints it's digest and the latest packages it could find via `apk` inside that container and prints out the names and versions.
Example output of `./get_pkg_versions.sh` for a new image, which is not yet pulled:
```plain
Unable to find image 'python:3.12.3-alpine3.18' locally
3.12.3-alpine3.18: Pulling from library/python
619be1103602: Pull complete
[...]
0eb61f1af52e: Pull complete
Digest: sha256:24680ddf8422899b24756d62b31eb5de782fbb42e9c2bb1c70f1f55fcf891721
Status: Downloaded newer image for python:3.12.3-alpine3.18
[Script output starts here]
Checking 5/5 latest package versions on python:3.12.3-alpine3.18
Image digest found: sha256:24680ddf8422899b24756d62b31eb5de782fbb42e9c2bb1c70f1f55fcf891721
---
build-base-0.5-r3
gcc-12.2.1_git20220924-r10
git-2.40.1-r0
openssh-keygen-9.3_p2-r1
ruby-3.2.4-r0
```
1. Copy the package versions and update the respective `ENV` with it manually in the `Dockerfile`. You also might add the digest to the base image.
1. Test building the image and you can commit it.

15
get_pkg_versions.sh Executable file
View file

@ -0,0 +1,15 @@
#!/bin/bash
set -euo pipefail
IFS=$'\n\t'
IMAGE=$(grep -oP 'FROM \K.*alpine[^ ]+' Dockerfile)
PACKAGES=$(grep -oP '#.+depName=alpine.+/\K[^ ]+' Dockerfile)
# shellcheck disable=SC2086
PACKAGES_NO_BR=$(echo ${PACKAGES} | tr -d '\n')
PACKAGES_VERSIONS=$(docker run --rm -t --entrypoint /bin/sh "$IMAGE" -c "apk --update --no-cache list $PACKAGES_NO_BR | cut -d ' ' -f 1 | grep -v '^fetch$'")
DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE" | cut -d '@' -f2)
echo "Checking $(echo "$PACKAGES" | wc -l)/$(echo "$PACKAGES_VERSIONS" | wc -l) latest package versions on $IMAGE"
echo "Image digest found: $DIGEST"
echo "---"
echo "$PACKAGES_VERSIONS"

View file

@ -1,8 +1,6 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"local>renovate/config"
],
"extends": ["local>renovate/config"],
"branchPrefix": "renovate-",
"groupName": "all dependencies",
"groupSlug": "all",
@ -10,18 +8,14 @@
{
"groupName": "all dependencies",
"groupSlug": "all",
"matchPackagePatterns": [
"*"
]
"matchPackageNames": ["/*/"]
}
],
"separateMajorMinor": false,
"customManagers": [
{
"customType": "regex",
"fileMatch": [
"^Dockerfile$"
],
"fileMatch": ["^Dockerfile$"],
"matchStrings": [
"#\\s*renovate:\\s*datasource=(?<datasource>.*?) depName=(?<depName>.*?)( versioning=(?<versioning>.*?))?\\sENV .*?_VERSION=\"(?<currentValue>.*)\"\\s"
],