1. 服务端

1.1. 开启CI

1.2. 配置CI

  1. CI中比较有用的变量 .https://docs.gitlab.com/ee/ci/variables/#syntax-of-environment-variables-in-job-scripts

echo "项目命名空间:$CI_PROJECT_NAMESPACE"
echo "项目名称:$CI_PROJECT_NAME"
echo "提交分支:$CI_COMMIT_REF_NAME"
echo "分支版本:$CI_COMMIT_REF_SLUG"
echo "项目地址:$CI_PROJECT_URL"
echo "提交人:$GITLAB_USER_LOGIN"
echo "提交ID:$CI_COMMIT_SHA"
echo "流水线ID:$CI_PIPELINE_ID"
echo "提交时间:$CI_COMMIT_TIMESTAMP"
echo "当前目录:$CI_PROJECT_DIR"
echo "当前目录:$CI_PROJECT_DIR"
http地址
部署脚本示例
deploy_path="/home/$CI_PROJECT_NAMESPACE/ci/$CI_PROJECT_NAME" #部署路径
ftp_path="/var/ftp/pub/ci/$(date +%Y%m%d)" #FTP发布路径
MAX_TIMEOUT=60 #停止服务最长等待时间
logging_file_path="/tmp/$CI_PROJECT_NAME.log" #日志文件路径

mvn package
if [ ! -f "$CI_PROJECT_DIR/target/$CI_PROJECT_NAME.jar" ]
then
   echo "打包失败!"
   exit 1
fi
if [ ! -d "$deploy_path" ]
then
   mkdir -p $deploy_path
fi
echo "部署至$deploy_path"
cp -f $CI_PROJECT_DIR/target/$CI_PROJECT_NAME.jar $deploy_path

if [ ! -d "$ftp_path" ]
then
   mkdir -p $ftp_path
fi
echo "发布至$ftp_path"
cp -f $CI_PROJECT_DIR/target/$CI_PROJECT_NAME.jar $ftp_path

cd $deploy_path # 需要先进入部署目录,否则java -jar 命令不会读取当前目录的配置

#停止
oldPid=$(ps ax | grep $CI_PROJECT_NAME | grep java | head -1 | awk '{print $1}')

if [ ${oldPid} ]; then
    echo '开始停止【'$CI_PROJECT_NAME'】服务'
    kill -15 $oldPid
fi

spa='.'
for((i=1;i<=$MAX_TIMEOUT;i++))
do
    # 等待1秒
    sleep 1
    printf "\r"
    oldPid=$(ps ax | grep $CI_PROJECT_NAME | grep java | head -1 | awk '{print $1}')
    if [ ${oldPid} ]; then
        printf "[%-${MAX_TIMEOUT}s] %d/%d秒" "$spa" "$i" "$MAX_TIMEOUT";
        spa+='.'
    else
        break
    fi
done
printf "\n"

if [ ${oldPid} ]; then
    echo '尝试'$MAX_TIMEOUT'秒优雅停止【'$CI_PROJECT_NAME'】服务失败,强制停止!!!'
    kill -9 $oldPid
else
    echo '成功停止【'$CI_PROJECT_NAME'】服务'
fi

oldPid=$(ps ax | grep $CI_PROJECT_NAME.jar | grep java | head -1 | awk '{print $1}')

if [ ${oldPid} ]; then
    echo 'App is running.'
    sleep 2
else
    echo '【'$CI_PROJECT_NAME'】服务停止成功!'
fi
rm -f $deploy_path/$CI_PROJECT_NAME.pid

#启动
echo "开始启动【$CI_PROJECT_NAME.jar】..."
if [[ $CI_PROJECT_NAME = "batch" ]];then
  # springboot2.2 日志配置有调整
  JAVA_OPTS="--spring.profiles.active=local --logging.file.name=$logging_file_path"
else
  JAVA_OPTS="--spring.profiles.active=local --logging.file=$logging_file_path"
fi
JVM_OPTS="-Xmx400m"
nohup java $JVM_OPTS -jar $deploy_path/$CI_PROJECT_NAME.jar $JAVA_OPTS >/dev/null 2>&1 &
echo $! > $deploy_path/$CI_PROJECT_NAME.pid
sleep 1
if [ -f "$logging_file_path" ]; then
  echo '正在检测日志【'$USER'@'$(hostname --fqdn)':'$logging_file_path'】...'
  { sed /"Tomcat started on port"/q; kill $!; } < <(exec timeout 8m tail -Fn 0 "$logging_file_path")
else
  echo "日志$logging_file_path不存在"
fi
# 重启可执行jar包
pkill $CI_PROJECT_NAME.jar
nohup $deploy_path/$CI_PROJECT_NAME.jar $JAVA_OPTS >/dev/null 2>&1 &

# 重启不可执行jar包
#ps -ef | grep $CI_PROJECT_NAME.jar | grep -v grep | cut -c 9-15 | xargs kill
#nohup java $JVM_OPTS -jar $deploy_path/$CI_PROJECT_NAME.jar $JVM_OPTS >/dev/null 2>&1 &
  1. 前端工程 由于npm每次都需要执行 npm install 下载依赖包,所以在配置gitlab-ci.yml的时候把 node_modules/ 目录缓存一下,如下示例:

stages:
- deploy
deploy:
  stage: deploy
  cache:
    paths:
      - node_modules/
  script:
    - angular_npm_publish.sh
  only:
    - master
  tags:
    - deploy

angular依赖发布脚本示例:

angular_npm_publish.sh
sed -i 's/"peerdependencies"/"dependencies"/g' package.json
npm install
sed -i 's/"dependencies"/"peerdependencies"/g' package.json
npm run packagr
npm publish

2. 客户端

2.1. 安装gitlab-runner

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.rpm.sh | sudo bash
yum install gitlab-ci-multi-runner

2.2. 配置gitlab-runner

vi /etc/gitlab-runner/config.toml

concurrent = 10
check_interval = 0

[[runners]]
  name = "name"
  url = "http://192.168.1.1/ci"
  token = ""
  executor = "shell"
  [runners.cache]

2.3. 常见问题:

  1. 访问偶尔出现Forbidden 原因:Gitlab使用rack_attack做了并发访问的限制。 解决方案:将Gitlab的IP设置为白名单即可。编辑配置文件:

vi /etc/gitlab/gitlab.rb 输入“/”,查找gitlab_rails['rack_attack_git_basic_auth']关键词,取消注释。 修改ip_whitelist白名单属性,加入Gitlab部署的IP地址。如下所示:

gitlab_rails['rack_attack_git_basic_auth'] = {
    'enabled' => true, # 启用/禁用 Rack Attack
    'ip_whitelist' => ["127.0.0.1","192.168.1.1"], # 白名单地址,多个IP用","隔开
    'maxretry' => 300, # 限制每个IP尝试登陆的次数
    'findtime' => 5, # 5秒后重置每IP的授权计数器
    'bantime' => 60 # 对多次错误登陆的IP封禁1小时(3600秒)
}
  1. gitlab-runner无法运行docker

sudo chmod 666 /var/run/docker.sock
  1. gitlab占用内存过高

    • 减少unicorn进程数,设置方法:CPU核心数+1。

      /etc/gitlab/gitlab.rb
      unicorn['worker_processes'] = 3
    • Postgresql占用内存太多,限制一下其内存:

      /etc/gitlab/gitlab.rb
      postgresql['shared_buffers'] = "256MB"
    • 减少sidekiq并发 ./etc/gitlab/gitlab.rb

sidekiq['concurrency'] = 25
  1. gitlab占用硬盘占用过高

    /etc/gitlab/gitlab.rb
    # 修改备份文件位置
    gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"
    # 设置备份文件保留时间为7天(单位:秒)
    gitlab_rails['backup_keep_time'] = 604800

设置完成后重启

sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart
  1. gitlab备份失败,pg连接数不够

pg的连接数默认100,设置为500

+ ./etc/gitlab/gitlab.rb

postgresql['max_connections'] = 500
  1. gitlab设置为external_url有端口号和https时nginx会跟着改

    /etc/gitlab/gitlab.rb
    nginx['listen_port'] = 80
    nginx['listen_https'] = false

2.4. 异机备份

  1. 生产本机ssh备份秘钥

ssh-keygen -t rsa -C gitlab_back -f ~/.ssh/id_rsa_gitlab_backup
  1. 设置服务器备份公钥

cat ~/.ssh/id_rsa_gitlab_backup.pub | ssh root@192.168.1.xxx "cat >> ~/.ssh/authorized_keys"
如果服务器没有 .ssh 文件夹,则执行 cat ~/.ssh/id_rsa_gitlab_backup.pub | ssh root@192.168.1.xxx "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
  1. 设置本地ssh环境

cat >> ~/.ssh/config << EOF
Host backup
     HostName 192.168.1.xxx
     User root
     PreferredAuthentications publickey
     IdentityFile ~/.ssh/id_rsa_gitlab_backup
EOF
  1. 备份 ./usr/local/bin/gitlab_backup.sh

# 输出备份时间
echo "`date '+%FT%T'`开始远程备份..."
# 备份
rsync -rPz -e ssh /var/opt/gitlab/backups/ backup:/home/data/gitlab
echo "`date '+%FT%T'`远程备份完成。"
  1. 定时后台执行 .配置`crontab -e`

0 0 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create >> /tmp/gitlab_backup_create.log 2>&1 &
0 1 * * * sh /usr/local/bin/gitlab_backup.sh >> /tmp/gitlab_backup.log 2>&1 &

3. 前端打包

普通前端
stages:
- npm_publish
npm_publish:
  stage: npm_publish
  cache:
    paths:
      - node_modules/
  script:
    - npm-ng-publish.sh
  when: manual
  tags:
    - deploy
jar包前端
stages:
- npm_publish
- mvn_deploy
npm_publish:
  stage: npm_publish
  cache:
    paths:
      - frontend/node_modules/
  script:
    - cd frontend
    - npm-ng-publish.sh
  when: manual
  tags:
    - deploy
mvn_deploy:
  stage: mvn_deploy
  cache:
    paths:
      - frontend/node_modules/
  script:
    - sed -i 's/"peerDependencies"/"dependencies"/g' frontend/package.json
    - repo_public_deploy
  when: manual
  tags:
    - deploy