I. What Is a Package Manager?
A package manager is a tool for managing software packages and their dependencies.
In modern software development, a project usually does not rely entirely on code written by itself; it needs to use standard libraries, third-party libraries, frameworks, tools, and other components.
Manually downloading, configuring, and updating these components every time creates a lot of repetitive work and easily leads to issues such as version inconsistency, dependency conflicts, and environment differences.
Package managers exist to solve these problems.
It is usually responsible for:
- Searching for and retrieving packages.
- Installing and uninstalling packages.
- Resolving dependencies between packages.
- Managing package versions.
- Downloading and verifying packages.
- Updating project dependencies.
- Maintaining dependency manifests and lockfiles.
- Managing package repositories and publishing workflows.
1. What Is a Package?
A package is a software component organized in a certain format for distribution and reuse.
A package usually contains:
| Component | Description |
|---|---|
| Source code or binary files | The actual implementation of the package |
| Metadata | Information such as name, version, and author |
| Dependency declarations | Other packages required for running or building |
| Entry point information | Module entry point or program entry point |
| License | The licensing terms of the package |
| Verification information | Used to verify file integrity |
| Documentation | Installation instructions, usage, and API documentation |
For example, a Python project can be published as a package via PyPI, a JavaScript project can be published via npm, and a Rust library can be published via crates.io.
Note that a package is not necessarily the same as a library. A package may also contain command-line tools, compiler plugins, build tools, applications, or other resources.
2. Basic Workflow of a Package Manager
A package manager usually looks up the package repository based on the dependencies declared by the project, resolves version constraints, retrieves the packages, and installs them into the specified environment.
flowchart TD
A["Project declares dependencies"] --> B["Package manager reads configuration"]
B --> C["Query package repository"]
C --> D["Resolve version constraints"]
D --> E["Compute dependency relationships"]
E --> F["Download packages"]
F --> G["Verify integrity"]
G --> H["Install dependencies"]
H --> I["Update dependency records"]
I --> J["Project uses dependencies"]
style A fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style C fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
style E fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style I fill:#FFF5DE,stroke:#E8D29A,color:#334155
style J fill:#E5F4EC,stroke:#A3CBB5,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
During actual execution, a package manager may also handle caching, platform adaptation, build scripts, environment isolation, dependency conflicts, and other issues.
II. Package Managers for Major Languages
Different languages have different package management ecosystems. The way package managers are used, their configuration files, and their dependency resolution rules also differ.
1. Python: pip, uv, Poetry
Common package management tools in the Python ecosystem include:
| Tool | Primary Use |
|---|---|
| pip | Install and manage Python packages |
| venv | Create Python virtual environments |
| Poetry | Project dependency, environment, and build management |
| uv | High-performance Python package management and project tool |
| conda | Environment and package management, often used in scientific computing |
pip is the most common package installation tool for Python.
# Install a third-party library
python -m pip install requests
# Install a specified version
python -m pip install requests==2.32.3
# Uninstall a package
python -m pip uninstall requests
# List installed packages
python -m pip list
# Show dependency information
python -m pip show requests
# Export packages in the current environment
python -m pip freeze > requirements.txt
Python projects typically use requirements.txt to record dependencies, and can also use pyproject.toml together with tools such as Poetry and uv to manage project configuration and dependencies.
It is important to distinguish: pip is mainly responsible for package installation, while venv is responsible for creating virtual environments. They are not the same tool.
2. Java: Maven, Gradle
The Java ecosystem mainly uses Maven and Gradle to manage dependencies and build tasks.
| Tool | Characteristics |
|---|---|
| Maven | Uses XML configuration, with clear conventions and a mature ecosystem |
| Gradle | Supports Groovy and Kotlin DSL, with flexible build logic |
Maven uses pom.xml to declare projects and dependencies.
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>33.3.1-jre</version>
</dependency>
</dependencies>
Gradle can use Kotlin DSL:
dependencies {
implementation("com.google.guava:guava:33.3.1-jre")
testImplementation("org.junit.jupiter:junit-jupiter:5.11.0")
}
Maven and Gradle are not only dependency management tools; they also handle project building, testing, packaging, and publishing.
3. JavaScript: npm, Yarn, pnpm
The JavaScript ecosystem mainly manages npm packages through package managers.
Common tools:
| Tool | Characteristics |
|---|---|
| npm | The default, common tool in the Node.js ecosystem, with a mature ecosystem |
| Yarn | Provides dependency management and workspace capabilities |
| pnpm | Improves dependency reuse efficiency through content-addressable storage and linking mechanisms |
Common npm commands:
# Initialize project
npm init -y
# Install dependencies
npm install axios
# Install dev dependencies
npm install --save-dev typescript
# Uninstall dependencies
npm uninstall axios
# View dependency tree
npm ls
# Update dependencies that match version constraints
npm update
Common project files:
package.json: Project metadata, scripts, and dependency declarations.package-lock.json: Locks the npm dependency resolution results.node_modules: Directory of locally installed dependencies.
npm, Yarn, and pnpm use different lock file formats. Generally, you should maintain the corresponding lock file according to the package manager chosen for the project, and avoid having multiple tools arbitrarily modify the same dependency tree.
4. Go: Go Modules
Go uses Go Modules to manage project modules and dependencies.
Common commands:
# Initialize module
go mod init example.com/myapp
# Add or upgrade dependencies
go get github.com/gin-gonic/gin
# Tidy dependencies
go mod tidy
# View dependency relationships
go mod graph
# View module information
go list -m all
Main files:
| File | Purpose |
|---|---|
go.mod |
Declares the module path and dependency requirements |
go.sum |
Records checksum information for dependency content |
Go Modules manages dependencies using module versions and supports module proxies, checksum databases, and version selection mechanisms.
5. Rust: Cargo
Rust officially uses Cargo to manage projects, dependencies, and builds.
Common commands:
# Create project
cargo new myapp
# Add dependencies
cargo add serde
# Build project
cargo build
# Run tests
cargo test
# View dependency tree
cargo tree
# Update dependencies
cargo update
Main files:
| File | Purpose |
|---|---|
Cargo.toml |
Declares project metadata and dependencies |
Cargo.lock |
Locks dependency resolution results |
target/ |
Build artifact directory |
Cargo is responsible not only for downloading dependencies, but also for compiling, testing, building, and publishing.
6. C/C++: Conan, vcpkg
Dependency management in the C/C++ ecosystem is more complex than in many modern languages because it involves factors such as compilers, operating systems, processor architectures, ABI, and build systems.
Common tools:
| Tool | Key features |
|---|---|
| Conan | Cross-platform C/C++ dependency management |
| vcpkg | C/C++ package management tool maintained by Microsoft |
| CMake | Cross-platform build system that can also integrate dependency discovery and retrieval |
Conan example:
# Install Conan
pip install conan
# Install dependencies
conan install . --output-folder=build
vcpkg example:
# Install a specified library
vcpkg install fmt
# Install the library for a specified platform configuration
vcpkg install fmt:x64-windows
C/C++ projects especially need to ensure that the build configurations of dependencies are consistent, such as compiler version, runtime library, static or dynamic linking, and target platform architecture.
III. Core Concepts of Dependency Management
Dependency management refers to the process of declaring, resolving, installing, updating, locking, and maintaining the software components that a project depends on.
Package managers are important tools for carrying out dependency management, but dependency management itself is a broader engineering practice.
1. What Is a Dependency?
A dependency is an external component that a project needs when running, building, or testing.
For example, a web service may depend on:
- Web framework.
- Database driver.
- JSON serialization library.
- Logging component.
- Authentication component.
- Testing framework.
Dependencies can be categorized by purpose of use.
| Type | Meaning | Example |
|---|---|---|
| Runtime dependency | Required when the program is running | Web framework, database driver |
| Development dependency | Required during development | Code formatting tool |
| Test dependency | Required when running tests | pytest, JUnit |
| Build dependency | Required for compilation and packaging | Compiler plugin, code generator |
| Optional dependency | Required only when certain features are enabled | Image processing extension |
Different package managers do not use exactly the same names and semantics for dependency categories.
2. Direct Dependencies and Transitive Dependencies
A direct dependency is a dependency that the project explicitly declares.
A transitive dependency is another component that a direct dependency further requires.
For example:
flowchart TD
A["Business project"] --> B["Web framework"]
A --> C["Logging library"]
B --> D["HTTP utility library"]
B --> E["Serialization library"]
D --> F["Network infrastructure components"]
E --> G["Encoding utility library"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
style G fill:#FFF5DE,stroke:#E8D29A,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
In this example:
- The web framework and logging library are direct dependencies of the project.
- The HTTP utility library and serialization library are transitive dependencies of the web framework.
- The network infrastructure components and encoding utility library are deeper-level transitive dependencies.
Transitive dependencies can form a complex dependency tree.
Therefore, the number of components actually used by a project is usually much larger than the number directly declared in configuration files.
3. Dependency Graph
A dependency graph is a data structure that describes the dependency relationships between software packages.
Usually, each software package can be viewed as a node, and each dependency relationship as a directed edge.
For example:
flowchart TD
A["Application"] --> B["Library A"]
A --> C["Library B"]
B --> D["Library C"]
C --> D
D --> E["Library E"]
C --> F["Library F"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style E fill:#FFF5DE,stroke:#E8D29A,color:#334155
style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
A dependency graph can help developers:
- Understand the project's dependency structure.
- Identify duplicate dependencies.
- Analyze dependency conflicts.
- Determine the scope of impact of vulnerabilities.
- Assess the impact of upgrades.
- Identify components that are no longer used.
A dependency graph may be a directed acyclic graph (DAG), but different ecosystems handle cyclic dependencies differently. Some tools reject certain cyclic dependencies, while other systems allow specific forms of circular references.
4. Version Management and Semantic Versioning
Software packages need version numbers to distinguish different releases.
Version management is an important part of dependency management, because different versions may have API changes, behavior changes, and compatibility differences.
1. Semantic Versioning
Semantic versioning typically uses:
MAJOR.MINOR.PATCH
For example:
3.5.2
The three numbers represent the following, respectively:
| Part | Meaning | Example |
|---|---|---|
| MAJOR | Incompatible API changes | Upgrade from 3.x to 4.x |
| MINOR | Backward-compatible feature additions | Upgrade from 3.5 to 3.6 |
| PATCH | Backward-compatible bug fixes | Upgrade from 3.5.2 to 3.5.3 |
The goal of semantic versioning is to let developers understand the compatibility impact of changes from the version number.
However, it depends on project maintainers following the specification correctly, and it cannot guarantee that all packages are released strictly according to semantic versioning.
2. Pre-release Versions
Packages may also publish pre-release versions, for example:
1.0.0-alpha.1
1.0.0-beta.2
1.0.0-rc.1
1.0.0
Common stages:
- Alpha: an early test version; features may be incomplete.
- Beta: features are relatively complete, but issues may still exist.
- RC: a release candidate, usually close to the final release.
- Stable: the final stable release.
Production environments usually need to use pre-release versions with caution, unless the project explicitly needs to validate new features in advance.
3. Version Constraints
Dependency declarations usually do not simply specify a single version; they can also set an allowed version range.
Common syntax:
| Syntax | Common meaning |
|---|---|
1.2.3 |
Exact version; specific behavior depends on the tool |
>=1.2.3 |
At least the specified version |
<2.0.0 |
Below the specified version |
^1.2.3 |
Allows updates within the compatible range |
~1.2.3 |
Usually limited to a smaller update range |
* |
Matches a broad range |
For example:
{
"dependencies": {
"axios": "^1.7.0",
"lodash": "~4.17.0"
}
}
Under npm's semantic versioning rules:
^1.7.0usually allows updating to a compatible 1.x version.~4.17.0usually allows updating to a 4.17.x patch version.
Version constraint syntax and resolution rules vary by package manager; the rules of one tool cannot be directly applied to other ecosystems.
4. Version Conflicts
Dependency conflicts can occur when different dependencies impose incompatible version requirements on the same package.
For example:
flowchart TD
A["Project"] --> B["Component A"]
A --> C["Component B"]
B --> D["Shared Library 1.x"]
C --> E["Shared Library 2.x"]
D --> F["Inconsistent Version Constraints"]
E --> F
F --> G["Dependency Resolution Failure or Runtime Risk"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
style G fill:#FBE9E9,stroke:#D9AAAA,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
Dependency conflicts may manifest as:
- Unable to find a version that satisfies all constraints.
- Type or symbol incompatibility during compilation.
- Class or method not found at runtime.
- Behavioral differences between versions.
- Plugin incompatible with the host program.
Resolution methods include unifying versions, upgrading upstream dependencies, excluding conflicting dependencies, using compatible versions, or running different versions through isolation mechanisms.
V. Dependency Resolution Mechanism
Dependency resolution is one of the core functions of a package manager.
When a project declares multiple dependencies, the package manager needs to find a set of concrete package versions that satisfy the constraints.
1. Inputs to Dependency Resolution
Dependency resolution usually needs to consider:
- Direct dependencies of the project.
- Version constraints of each dependency.
- Transitive dependency declarations.
- Available versions in package repositories.
- Operating system and processor architecture.
- Optional features and platform conditions.
- Compatibility rules between dependencies.
2. Basic Process of Dependency Resolution
flowchart TD
A["Read Project Dependency Declarations"] --> B["Fetch Candidate Package Versions"]
B --> C["Read Transitive Dependencies"]
C --> D["Build Dependency Constraints"]
D --> E["Select Candidate Version"]
E --> F{"Are All Constraints Satisfied?"}
F -->|"Yes"| G["Generate Dependency Resolution Result"]
F -->|"No"| H["Try Other Version Combinations"]
H --> I{"Is There a Feasible Solution?"}
I -->|"Yes"| E
I -->|"No"| J["Report Dependency Conflict"]
G --> K["Download and Install"]
style A fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style C fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
style E fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style H fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style I fill:#FFF5DE,stroke:#E8D29A,color:#334155
style J fill:#FBE9E9,stroke:#D9AAAA,color:#334155
style K fill:#E5F4EC,stroke:#A3CBB5,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
This is the dependency resolution process at a conceptual level. Different package managers use different algorithms and may adopt backtracking search, constraint solving, version-first strategies, or other algorithms.
3. Version Selection Strategies
Package managers may adopt different version selection strategies:
Latest Compatible Version
Choose a newer version while satisfying the constraints.
Minimal Version Selection
Prefer lower versions that satisfy the requirements to reduce unnecessary upgrades.
Lockfile Priority
When a lockfile already exists, prefer the concrete versions already resolved in the lockfile.
Unified Version Strategy
Try to use a consistent version of the same dependency throughout the project.
Multiple Versions Coexisting
In ecosystems that allow it, the same project can use multiple versions of a package at the same time.
Strategies differ among package managers, and you cannot assume that all tools will automatically choose the latest version.
VI. Lockfiles and Reproducible Builds
Lockfiles are an important part of modern dependency management systems.
Dependency declarations usually describe the range of allowed versions, while a lockfile records the specific versions ultimately determined by a particular dependency resolution.
1. Why Are Lockfiles Needed?
Suppose the project declares:
{
"dependencies": {
"axios": "^1.7.0"
}
}
When installing at different times, different compatible versions may be resolved.
For example:
- When developer A installs, version 1.7.2 is resolved.
- When developer B installs, version 1.8.0 is resolved.
Although both versions satisfy the constraint, they may behave differently.
A lock file can record exact versions, reducing this discrepancy.
2. Common Lock Files
| Ecosystem | Lock File |
|---|---|
| Python | poetry.lock, uv.lock |
| JavaScript | package-lock.json, pnpm-lock.yaml, yarn.lock |
| Rust | Cargo.lock |
| Go | go.sum |
| Java | Maven and Gradle can use mechanisms such as dependency locking |
Note especially: go.sum mainly records checksum information for module contents and is not equivalent to npm's package-lock.json. Go's version selection also depends on module information such as go.mod.
3. Main Purposes of Lock Files
- Pin dependency resolution results.
- Reduce dependency differences between development and production environments.
- Improve the consistency of continuous integration builds.
- Facilitate dependency vulnerability audits.
- Record dependency upgrade changes.
- Support troubleshooting build issues.
4. Reproducible Build
Reproducible builds emphasize obtaining consistent build results under controlled conditions using the same source code, dependencies, and build environment.
A lock file is only an important part of achieving reproducible builds.
Other factors also need to be considered:
- Compiler version.
- Operating system and architecture.
- Build tool version.
- Environment variables.
- System-level dependencies.
- Timestamps and randomness.
- Build scripts and external resources.
A fully reproducible build cannot rely on a lock file alone.