Portal¶
Linux Bookmarks
Stuff related to GNU/Linux.
Install Linux
USB burning tools
The following tools make installation USB keys from ISO files.
- Ventoy : an open source tool to create a bootable USB drive loading ISO files in the partition.
- Rufus : flashing Linux/Windows ISOs in Windows.
- Etcher : flashing Linux ISOs in Linux/Windows.
- Gnome disks
Linux Wikis and forums
Command lines and shell scripting
- Awesome shell
- cheat.sh : cheat sheet for linux commands.
- Shell check
- Explain shell
- edX: introduction to Linux
- Learn Bash in Y minutes
- Ryan's Tutorial
Linux Input Methods
Input methods enable multilingual inputs, including CJK (Chinese, Japanese, and Korean).
Fcitx5
Fcitx is a lightweight input method framework that provides environment-independent language support for Linux. The development energy mainly focuses on the release of the new version 5.
Install
command -v apt && sudo apt install fcitx5 fcitx5-chewing fcitx5-material-color
command -v pacman && sudo pacman -S fcitx5-im fcitx5-chewing fcitx5-material-color
Setup
Add the following lines to ~/.xprofile
or ~/.profile
if fcitx
does not load on startup.
export INPUT_METHOD=fcitx5
export GTK_IM_MODULE=fcitx5
export QT_IM_MODULE=fcitx5
export XMODIFIERS=\@im=fcitx5
Ibus
ibus is an input method framework using DBUS. ibus
integrates better with the GNOME desktop environment.
Install
command -v apt && sudo apt install ibus ibus-chewing
command -v pacman && sudo pacman -S ibus ibus-chewing
Setup
Add these lines to ~/.xprofile
or ~/.profile
if ibus
does not load on startup.
Linux Themes
Papirus icon theme and Kvantum SVG engine
- PapirusDevelopmentTeam/papirus-icon-theme
- tsujan/Kvantum : SVG-based theme engine
Install (Ubuntu)
sudo add-apt-repository -y ppa:papirus/papirus # Papirus icon theme
sudo apt update && sudo apt install -y papirus-icon-theme qt5-style-kvantum qt5ct
Materia theme
A clean theme for both KDE and Gnome.
Install (Ubuntu)
sudo add-apt-repository -y ppa:papirus/papirus
sudo apt update && sudo apt install materia-gtk-theme materia-kde
Set fonts
Fonts for materia theme:
- To properly display the theme, use a font family that includes Medium weight (e.g.
Roboto
or M+). - Set the font size to 9.75 (= 13px at 96dpi) or 10.5 (= 14px at 96dpi).
Qogir theme
vinceliuice/Qogir-theme is a flat Design theme for GTK.
Install (Ubuntu)
Download and run the install script from vinceliuice/Qogir-theme.
Change Linux user directory
Use sudo
and at
to schedule the usermod
command, which changes the user's home dir.
In the at
interface
pkill -u UID # kill user processes
usermod -m -d /new/home # Change user home dir (-d) and move (-m) the content into the new folder
Ctrl+D
to exit the at
interface. Logout, wait 10 minutes, and login.
See also
heredoc: Passing multiple lines of string
Use heredoc to pass the string as-is between two delimiters (e.g. EOF
)
cat << "EOF" >> ~/.xprofile
# ~/.xprofile
export GTK_IM_MODULE=ibus
export QT_IM_MODULE=ibus
export XMODIFIERS=@im=ibus
ibus-daemon -drx
EOF
Will append the following lines to ~/.xprofile
:
Reload nvidia GPU driver
Reload nvidia GPU driver to fix "NVML: Driver/library version mismatch" error without rebooting. Source
GitLab
GitLab CI/CD is a tool built into GitLab for software development for Continuous Integration (CI) and Continuous Delivery/Deployment (CD).
Parallel Matrix build
Test and build in parallel with matrix build in Gitlab CI/CD.
For example,
test:
image: $IMAGE
script:
- echo $MSG
- python -V
parallel:
matrix:
# First cartesian set of parameters
- IMAGE: ['python:3.6-alpine', 'python:3.7-alpine']
MSG: ['Test1', 'Test2']
# Second cartesian set of parameters
This will create 4 jobs with a combination of a custom message and a specific Python image.
See also the blog post by Michael Friedrich for more parallel matrix build with GitLab CI/CD.
Replace old only/except
with new rules
to include or exclude jobs in pipelines
Note
Rules cannot be used together with only/except. Otherwise, GitLab will return a key may not be used with rules
error.
only run if this is a scheduled pipeline
scheduled-update:
# only run if this is a scheduled pipeline
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
Run upon push
Run upon merge request
Run only for the commits in the default branch
Run only for tags
Choose a specific runner
Use tags
to tun jobs in a specific runner e.g., your self-hosted GitLab runner in the workstation.
Create a release
Create a release with GitLab CI/CD pipelines with the release-cli
docker image:
release_job:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli:latest
rules:
- if: $CI_COMMIT_TAG # Run this job when a tag is created manually
script:
- echo "Running the release job."
release:
name: "Release $CI_COMMIT_TAG"
description: "Release created using the release-cli."
Cache Conda Packages
We can cache conda packages by setting CONDA_PKGS_DIRS
environment variable inside the project folder (CI_PROJECT_DIR
) so that the GitLab runner can cache these dependencies.
image: condaforge/miniforge3:latest
variables:
CONDA_PKGS_DIRS: "${CI_PROJECT_DIR}/.cache/conda/pkgs"
cache:
- key:
files:
- environment.yml
paths:
- .env/
- .cache/conda/pkgs
before_script:
- conda env update --prefix ./.env --file environment.yml --prune
- source activate ./.env
Because GitLab only caches files inside the project folder (CI_PROJECT_DIR
)
CONDA_PKGS_DIRS
is set to${CI_PROJECT_DIR}/.cache/conda/pkgs
to hold the downloaded compressed packages.- Extracted environment folder is set to
${CI_PROJECT_DIR}/.env
using the--prefix
option.
Conda will create the runtime environment according to environment.yml
. The environment folder will be created (if not present) or cached. The option --prune
means conda will remove unnecessary packages for subsequent caching.
Git Operations in GitLab CI/CD
Using SSH keys
Warning
Currently the private key cannot be masked and base64 encoding/decoding is needed.
You can use a pair of SSH keys to access a git repository
- The private key would be a CI/CD project variable
- The public key would be a deploy key
You also need additional steps to setup a SSH client in the pipeline.
before_script:
# apt-get applies to Debian-based images. Change the package manager if needed.
- 'which ssh-agent || ( apt-get update -qy && apt-get install openssh-client -qqy )'
- 'which git || ( apt-get update -qy && apt-get install git -qqy )'
- eval `ssh-agent -s`
- echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null # add ssh key
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
And replace the default HTTP-based git origin with the SSH one.
Using a personal access token (PAT)
Compared to SSH, using a personal access token (PAT) with write repo
right might be simpler. In the following example, the PAT is stored as a masked CI/CD variable GIT_PUSH_TOKEN
.
script:
- bash update.sh
- |
if [ -n $(git status --porcelain) ]; then
echo "Committing updates"
git config --global user.name "${GITLAB_USER_NAME}"
git config --global user.email "${GITLAB_USER_EMAIL}"
git add .
git commit -m "Automated update: $(date '+%Y-%m-%d-%H-%M-%S')"
git push "https://${GITLAB_USER_NAME}:${GIT_PUSH_TOKEN}@${CI_REPOSITORY_URL#*@}"
exit;
else
echo "no change, nothing to commit"
fi
For a MR pipeline, GitLab provides git push
options for merge request settings.
script:
- bash update.sh
- |
if [ -n $(git status --porcelain) ]; then
echo "Committing updates"
NEW_BR=auto-update-$(date '+%Y-%m-%d-%H-%M-%S')
git config --global user.name "${GITLAB_USER_NAME}"
git config --global user.email "${GITLAB_USER_EMAIL}"
git checkout -b ${NEW_BR}
git add .
git commit -m "${NEW_BR}"
git push "https://${GITLAB_USER_NAME}:${GIT_PUSH_TOKEN}@${CI_REPOSITORY_URL#*@}" \
-o merge_request.create \
-o merge_request.target="${CI_DEFAULT_BRANCH}" \
-o merge_request.merge_when_pipeline_succeeds \
-o merge_request.remove_source_branch \
-o merge_request.title="${NEW_BR}" \
-o merge_request.label="automated update" \
-o merge_request.assign="${GITLAB_USER_NAME}"
exit;
else
echo "no change, nothing to commit"
fi
Synchronize GitLab repo to GitHub
Assuming you have two identical repositories on GitLab and GitHub each (you can do this by importing one's repo to the other), the following steps show how to mirror GitLab repositories to GitHub with deploy SSH keys.
On the GitLab side
- In the GitLab repo, go to
Settings
/Repository
/Mirroring repositories
and setGit repository URL
asssh://git@github.com/<namespace>/<repo>.git
. e.g.ssh://git@github.com/sosiristseng/docker-python-julia.git
Warning
The GitHub button gives git@github.com:<namespace>/<repo>.git
as the repo URL, one should change it to ssh://git@github.com/<namespace>/<repo>.git
for GitLab to access the repository.
-
Set
Mirror direction
to push.
-
Set
Authentication method
to SSH public key. Optionally you can clickDetect host keys
.
-
(Optionally) check "Keep divergent refs" to prevent force pushes and/or "Mirror only protected branches" for a cleaner GitHub mirror.
- Click
Mirror repository
. - Copy the SSH public key (the middle button) and go to the GitHub mirror repo.
On the GitHub side
In the Github mirror repository, go to Settings
/Deploy keys
and add deploy key.
Paste the SSH public key copied from the GitLab source. Give it a title, allow write access, click add key
to finish this step, and viola.