CI Proxy Cache

2026-01-25 1559 words 4 minutes

Contents

CI运行时，每次从公网下载太慢了，但是如果想把所有的依赖都镜像了，那就太大了，尤其是pypi镜像， 10TB都存不下。

nexus3

services:
  nexus:
    image: sonatype/nexus3:3.87.1-alpine
    container_name: nexus
    restart: always
    volumes:
      - /srv/nexus-data:/nexus-data
    environment:
      - VIRTUAL_HOST=your.domain.com
      - VIRTUAL_PORT=8081
      - INSTALL4J_ADD_VM_PARAMS=-Xms4g -Xmx4g -XX:MaxDirectMemorySize=4g

pypi

新建pypi仓库，可以选择取个名字，比如pypi，Remote storage填 https://pypi.tuna.tsinghua.edu.cn

cargo

community的nexus3对cargo支持不太好，直接新建cargo仓库是不好使的。

cargo的仓库分两部分，index和dl，可以使用多个raw的仓库，来处理。

cargo-config：raw hosted，仅存储修改过的config.json {"dl":"https://your.domain.com/repository/cargo-dl/crates","api":"https://your.domain.com/repository/cargo"}
cargo-index：raw proxy，Remote storage填 https://mirrors.bfsu.edu.cn/crates.io-index/
cargo: raw group，cargo-config和cargo-index，cargo-config排到前面
cargo-dl：raw proxy，Remote storage填 https://rsproxy.cn/api/v1/

npm镜像

npm镜像也可以使用nexus3，因为有个别package不好使，然后换了verdaccio，不过 verdaccio 也同样有个别package不好使，只能手动下载下来，放到404的url位置，比nexus3简单一点， nexus3需要再搞一个hosted，再搞个group

services:
  verdaccio:
    image: verdaccio/verdaccio:6.2
    restart: always
    environment:
      - VIRTUAL_HOST=your.domain.com
      - VIRTUAL_PORT=4873
      - HTTPS_METHOD=noredirect
      - HSTS=off
    volumes:
      - ./verdaccio_conf:/verdaccio/conf
      - /srv/verdaccio/storage:/verdaccio/storage

verdaccio_conf文件夹里需要创建config.yaml文件，内容如下：

#
# This is the default configuration file. It allows all users to do anything,
# please read carefully the documentation and best practices to
# improve security.
#
# Look here for more config file examples:
# https://github.com/verdaccio/verdaccio/blob/master/docker-examples/README.md
#
# Read about the best practices
# https://verdaccio.org/docs/best

# Path to a directory with all packages
storage: /verdaccio/storage/data

# Path to a directory with plugins to include, the plugins folder has the higher priority for loading plugins
# Disable this folder to avoid warnings if is not used
plugins: /verdaccio/plugins

# Web UI settings
# https://verdaccio.org/docs/webui
web:
  title: Verdaccio
  # Disable complete web UI
  # enabled: false
  # Custom colors for header background and font
  # primaryColor: "#4b5e40"
  # Custom logos and favicon
  # logo: ./path/to/logo.png
  # logoDark: ./path/to/logoDark.png
  # favicon: ./path/to/favicon.ico
  # Disable gravatar support
  # gravatar: false
  # By default, packages are ordered ascending
  # sort_packages: asc | desc
  # Convert your UI to the dark side
  # darkMode: true
  # html_cache: true
  # By default, all features are displayed
  # login: true
  # showInfo: true
  # showSettings: true
  # In combination with darkMode you can force specific theme
  # showThemeSwitch: true
  # showFooter: true
  # showSearch: true
  # showRaw: true
  # showDownloadTarball: true
  # showUplinks: true
  #
  # HTML tags injected before ends </head>
  # metaScripts:
  #   - '<script type="text/javascript" src="https://code.jquery.com/jquery-3.5.1.slim.min.js"></script>'
  #   - '<script type="text/javascript" src="https://browser.sentry-cdn.com/5.15.5/bundle.min.js"></script>'
  #   - '<meta name="robots" content="noindex">'
  #
  # HTML tags injected as first child in <body>
  # scriptsBodyBefore:
  #   - '<div id="myId">html before webpack scripts</div>'
  #
  # HTML tags injected as last child in </body>
  # scriptsBodyAfter:
  #   - '<script type="text/javascript" src="https://my.company.com/customJS.min.js"></script>'
  #
  # Public path for template manifest scripts (only manifest)
  # publicPath: http://somedomain.org/

# Settings for authentication plugin
# https://verdaccio.org/docs/configuration#authentication
auth:
  htpasswd:
    file: /verdaccio/storage/htpasswd
    # Maximum amount of users allowed to register, defaults to "+inf".
    # You can set this to -1 to disable registration.
    # max_users: 1000

# A list of other known repositories we can talk to
# https://verdaccio.org/docs/configuration#uplinks
uplinks:
  npmjs:
    url: https://registry.npmmirror.com/

# Learn how to protect your packages
# https://verdaccio.org/docs/protect-your-dependencies/
# https://verdaccio.org/docs/configuration#packages
packages:
  '@*/*':
    # scoped packages
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: npmjs

  '**':
    # allow all users (including non-authenticated users) to read and
    # publish all packages
    #
    # you can specify usernames/groupnames (depending on your auth plugin)
    # and three keywords: "$all", "$anonymous", "$authenticated"
    access: $all

    # allow all known users to publish/unpublish packages
    # (anyone can register by default, remember?)
    publish: $authenticated
    unpublish: $authenticated

    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: npmjs

# To improve your security configuration and avoid dependency confusion
# consider removing the proxy property for private packages
# https://verdaccio.org/docs/best#remove-proxy-to-increase-security-at-private-packages

# https://verdaccio.org/docs/configuration#server
# You can specify HTTP/1.1 server keep alive timeout in seconds for incoming connections.
# A value of 0 makes the http server behave similarly to Node.js versions prior to 8.0.0, which did not have a keep-alive timeout.
# WORKAROUND: Through given configuration you can workaround following issue https://github.com/verdaccio/verdaccio/issues/301. Set to 0 in case 60 is not enough.
server:
  keepAliveTimeout: 60
  # The pluginPrefix replaces the default plugins prefix which is `verdaccio`. Please don't include `-`. If `something` is provided
  # the resolved package will be `something-xxxx`.
  # pluginPrefix: something
  # A regex for the password validation /.{3}$/ (3 characters min)
  # An example to limit to 10 characters minimum
  # passwordValidationRegex: /.{10}$/
  # Allow `req.ip` to resolve properly when Verdaccio is behind a proxy or load-balancer
  # https://expressjs.com/en/guide/behind-proxies.html
  # trustProxy: '127.0.0.1'

# https://verdaccio.org/docs/configuration#offline-publish
# publish:
#   allow_offline: false
#   check_owners: false
#   keep_readmes: 'latest' | 'tagged' | 'all'

# Define public URL of registry in combination with VERDACCIO_PUBLIC_URL environment variable
# https://verdaccio.org/docs/configuration#url-prefix
# url_prefix: /verdaccio/
#
# Examples:
# VERDACCIO_PUBLIC_URL='https://somedomain.org'
# url_prefix: '/my_prefix'
# // url -> https://somedomain.org/my_prefix/
#
# VERDACCIO_PUBLIC_URL='https://somedomain.org'
# url_prefix: '/'
# // url -> https://somedomain.org/
#
# VERDACCIO_PUBLIC_URL='https://somedomain.org/first_prefix'
# url_prefix: '/second_prefix'
# // url -> https://somedomain.org/second_prefix/

# Security settings
# https://verdaccio.org/docs/configuration#security
# security:
#   api:
#     legacy: true
#     jwt:
#       sign:
#         expiresIn: 29d
#       verify:
#         someProp: [value]
#   web:
#     sign:
#       expiresIn: 1h # 1 hour by default
#     verify:
#       someProp: [value]

# https://verdaccio.org/docs/configuration#user-rate-limit
# userRateLimit:
#   windowMs: 50000
#   max: 1000

# https://verdaccio.org/docs/configuration#max-body-size
# max_body_size: 10mb

# https://verdaccio.org/docs/configuration#listen-port
# listen:
#   - localhost:4873            # default value
#   - http://localhost:4873     # same thing
#   - 0.0.0.0:4873              # listen on all addresses (INADDR_ANY)
#   - https://example.org:4873  # if you want to use https
#   - "[::1]:4873"              # ipv6
#   - unix:/tmp/verdaccio.sock  # unix socket

# The HTTPS configuration is useful if you do not consider use a HTTP Proxy
# https://verdaccio.org/docs/configuration#https
# https:
#   key: ./path/verdaccio-key.pem
#   cert: ./path/verdaccio-cert.pem
#   ca: ./path/verdaccio-csr.pem

# https://verdaccio.org/docs/configuration#proxy
# http_proxy: http://something.local/
# https_proxy: https://something.local/
# no_proxy: localhost,127.0.0.1,server.local

# https://verdaccio.org/docs/configuration#notifications
# notify:
#   method: 'POST'
#   headers: '[{ "Content-Type": "application/json" }]'
#   endpoint: 'https://usagge.hipchat.com/v2/room/3729485/notification?auth_token=mySecretToken'
#   content: '{"color":"green","message":"New package published: * {{ name }}*","notify":true,"message_format":"text"}'

# Settings for middleware plugins
# https://verdaccio.org/docs/plugins#middleware-configuration
middlewares:
  audit:
    enabled: true
    # timeout: 10000

# Log settings
# https://verdaccio.org/docs/logger
# Redaction: https://getpino.io/#/docs/redaction
# Synchronous logging: https://getpino.io/#/docs/asynchronous
log:
  type: stdout
  format: pretty
  level: http
#  redact:
#    paths: ['req.header.authorization','req.header.cookie','req.remoteAddress','req.remotePort','ip','remoteIP','user','msg']
#    censor: '<redacted>'
#  sync: true

# Feature flags (experimental settings that can be changed or removed in the future)
# https://verdaccio.org/docs/configuration#experiments
# experiments:
#  # Support for npm token command
#  token: false
#  # Enable tarball URL redirect for hosting tarball with a different server.
#  # The tarball_url_redirect can be a template string
#  tarball_url_redirect: 'https://mycdn.com/verdaccio/${packageName}/${filename}'
#  # The tarball_url_redirect can be a function, takes packageName and filename and returns the url,
#  # when working with a js configuration file
#  tarball_url_redirect(packageName, filename) {
#    const signedUrl = // generate a signed url
#    return signedUrl;
#  }
# Renamed from "experiments" to "flags" in next major release
# flags:
#  changePassword: true
#  searchRemote: true

# Translate your registry, API and web UI
# List of the available translations https://github.com/verdaccio/verdaccio/blob/master/packages/plugins/ui-theme/src/i18n/ABOUT_TRANSLATIONS.md
i18n:
  web: en-US

重要的是

uplinks:
  npmjs:
    url: https://registry.npmmirror.com/