Git 嵌套子模块和依赖项

Git nested submodules and dependencies

提问by Mauricio Scheffer

Let's say I have four projects named Core, A, B, Super. The dependency tree is like this:

假设我有四个名为Core、A、B、Super 的项目。依赖树是这样的：

Super ---> Core
       |-> A -> Core
       |-> B -> Core

I want each project to be stand-alone, that is, I want to be able to check-out and compile each project on its own (each with its dependencies of course).

我希望每个项目都是独立的，也就是说，我希望能够自行检出和编译每个项目（当然每个项目都有其依赖项）。

I thought about mapping each project to a repository and then referring dependencies with submodules, but I see the following issues with that approach:

我想过将每个项目映射到存储库，然后使用子模块引用依赖项，但我发现该方法存在以下问题：

1. When checking out Superwith all its dependencies, I'd end up with three copies of Core.
2. Since submodules are fully independent, each of these three copies could be pointing to different revisions of Coreand that would be a mess.

1. 当检查Super及其所有依赖项时，我最终会得到Core 的三个副本。
2. 由于子模块是完全独立的，这三个副本中的每一个都可能指向Core 的不同版本，这将是一团糟。

So... Am I missing something? Did I misunderstand git submodules or misusing them? Is there any other solution to this problem (other than resorting to binary dependencies)?

所以......我错过了什么吗？我是否误解了 git 子模块或误用了它们？这个问题有没有其他解决方案（除了诉诸二进制依赖）？

采纳答案by VonC

You just discovered the lack of overridden dependencies with Git submodules:

您刚刚发现 Git 子模块缺少覆盖依赖项：

If Super depends on Core, its dependency of Core should "override" the ones A and B have with Core.

如果 Super 依赖于 Core，那么它对 Core 的依赖应该“覆盖”A 和 B 对 Core 的依赖。

The only way to emulate that would be to create your Super project the way you did,
andto remove the sub-module Core of A and B.
(meaning Super depends now on A' and B', A' being A without Core, B' being B without Core)

模拟它的唯一方法是按照你的方式创建你的超级项目，
并删除 A 和 B 的子模块核心。
（意思是超级现在依赖于 A' 和 B'，A' 是没有核心的 A， B'是没有核心的B）

回答by Igor Zevaka

git repositories should be fairly atomic in the way that each repository is a stand-alone entity for a specific purpose. What is the purpose of the super project other than combining projects A and B? If there isn't anything unique (i.e. files that are not in either A, B or Core) then it's fairly redundant.

git 存储库应该是相当原子的，因为每个存储库都是用于特定目的的独立实体。超级项目除了结合A和B项目还有什么目的？如果没有任何独特的东西（即不在 A、B 或 Core 中的文件），那么它就是相当多余的。

EDIT: Because git submodules are especially painful at one place that I worked at we set up our own dependency system that tracks dependent repos via text files. We set it up so it always tracks the head of a branch, not a particular commit.

编辑：因为 git 子模块在我工作的一个地方特别痛苦，所以我们建立了自己的依赖系统，通过文本文件跟踪依赖的 repos。我们将它设置为始终跟踪分支的头部，而不是特定的提交。

We were able to set up all our projects as though they are a part of the Super project like this:

我们能够像这样设置我们所有的项目，就好像它们是 Super 项目的一部分一样：

Super
|-A
|-B
|-Core

The projects will reference each other using relative paths e.g. ../A/include.h. Checking out repo A will not work, you would have to create another "super" repo for working just on A:

这些项目将使用相对路径相互引用，例如../A/include.h. 签出 repo A 将不起作用，您必须创建另一个“超级” repo 来仅在 A 上工作：

AWorking
|-A
|-Core

EDITAnother reason for this behaviour in git is that it can't track things that are above the root repo directory(i.e. above the folder containing .git folder), which would definitely be required if you want your super-projects and sub-projects refer to the same repositories.

编辑git 中这种行为的另一个原因是它无法跟踪根 repo 目录上方的内容（即包含 .git 文件夹的文件夹上方），如果您想要超级项目和子项目，这肯定是必需的引用相同的存储库。

回答by Charlie

I think the issue here is a mismatch between the design of Git and the problem you are looking to solve.

我认为这里的问题是 Git 的设计与您要解决的问题之间的不匹配。

Git is good for keeping track of Trees. Dependency relationships between projects can (and likely do) form a Graph. A Tree is a Graph but a Graph is not necessarily a Tree. Since your problem is how to effectively represent a Graph, a Tree is not the best tool for the job.

Git 非常适合跟踪树。项目之间的依赖关系可以（并且可能确实）形成一个图。树是图，但图不一定是树。由于您的问题是如何有效地表示图形，因此树不是完成这项工作的最佳工具。

Here's an approach that might work:

这是一种可能有效的方法：

A git project has a .gitmodules directory where it records "hints" stating which projects a commit may depend on, where they can be found, and what path inside the project they are expected to be inserted at. ( http://osdir.com/ml/git/2009-04/msg00746.html)

一个 git 项目有一个 .gitmodules 目录，其中记录了“提示”，说明提交可能依赖于哪些项目，可以在哪里找到它们，以及它们应该插入到项目中的哪个路径。( http://osdir.com/ml/git/2009-04/msg00746.html)

You could add a script which reads this information from a set of projects, maps the hints found in each project's .gitmodules file to the locations on the filesystem where those projects have actually been placed, and then adds symbolic links from the paths where git expects to check out submodules to the actual filesystem locations of the respective projects.

您可以添加一个脚本，该脚本从一组项目中读取此信息，将在每个项目的 .gitmodules 文件中找到的提示映射到文件系统上实际放置这些项目的位置，然后从 git 期望的路径添加符号链接将子模块检出到相应项目的实际文件系统位置。

This approach uses symbolic links to break out of the Tree mold and build a Graph. If we record the links directly in git repos, we'd have relative paths specific to our local setup recorded in the individual projects, and the projects wouldn't be 'fully independent' like you wanted. Hence, the script to dynamically build the symlinks.

这种方法使用符号链接打破树模型并构建图形。如果我们直接在 git repos 中记录链接，我们将在各个项目中记录特定于我们本地设置的相对路径，并且这些项目不会像您想要的那样“完全独立”。因此，动态构建符号链接的脚本。

I'm thinking this approach might interfere with git in undesirable ways, since we've taken paths where it expects to find one thing, and put something else there instead. Maybe we could .gitignore the symlink paths. But now we're writing those paths down twice and violating DRY. At this point we've also gotten pretty far away from pretending to use submodules. We could record the dependencies elsewhere in each project, and leave the .gitmodules file for the things git expects. So we'll make up our own file, say, .dependencies, and each project can state its dependencies there. Our script will look there and then go and build its symlinks.

我认为这种方法可能会以不受欢迎的方式干扰 git，因为我们已经采取了它期望找到一件事的路径，并在那里放了其他东西。也许我们可以 .gitignore 符号链接路径。但是现在我们将这些路径写了两次并且违反了 DRY。在这一点上，我们也离假装使用子模块很远了。我们可以在每个项目的其他地方记录依赖关系，并将 .gitmodules 文件留给 git 期望的东西。因此，我们将创建自己的文件，例如 .dependencies，并且每个项目都可以在那里声明其依赖项。我们的脚本会在那里查看，然后去构建它的符号链接。

Hmm, I think I may have just described an ad-hoc package management system, with its own lightweight package format :)

嗯，我想我可能刚刚描述了一个特殊的包管理系统，它有自己的轻量级包格式:)

megamic's suggestion seems like a good use of git submodules to me. We're only dealing with keeping track of a Set here rather than a Graph, and a Set fits easily into a Tree. A Tree one level deep is essentially a parent node and a Set of child nodes.

megamic 的建议对我来说似乎很好地使用了 git 子模块。我们在这里只处理跟踪 Set 而不是 Graph，并且 Set 很容易放入 Tree 中。一层深的树本质上是一个父节点和一组子节点。

As you pointed out, that does not completely solve the problem stated in your question. We can break out two distinct types of "this works with that" information we're likely interested in: 1. A statement from a version of a project (presumably by the project's author) saying "I require version X of project Y" 2. A statement used by your own build setup saying "I've successfully tested our whole system using this set of project versions"

正如您所指出的，这并不能完全解决您问题中所述的问题。我们可以将我们可能感兴趣的两种不同类型的“这与那个”信息分开： 1. 来自项目版本的声明（大概是项目的作者）说“我需要项目 Y 的版本 X” 2 . 您自己的构建设置使用的声明，说“我已经使用这组项目版本成功测试了我们的整个系统”

megamic's answer solved (2) but for (1) we still want projects to tell us what their dependencies are. Then we can use the info from (1) to compute those version sets which we'll end up recording as (2). This is a complex enough problem to warrant its own tool, which brings us back to package management systems :)

megamic 的回答解决了 (2) 但对于 (1) 我们仍然希望项目告诉我们它们的依赖项是什么。然后我们可以使用来自 (1) 的信息来计算我们最终将记录为 (2) 的版本集。这是一个足够复杂的问题，需要它自己的工具，这让我们回到包管理系统:)

As far as I know, most of the good package management tools are made for users of a specific language or operating system. See Bundler for 'gem' packages in the ruby world and apt for '.deb' packages in the Debian world.

据我所知，大多数好的包管理工具都是为特定语言或操作系统的用户制作的。参见 Bundler 了解 ruby​​ 世界中的“gem”包，以及 Debian 世界中“.deb”包的 apt。

If anyone knows of a good language-neutral, OS-neutral solution to this that is well-suited to 'polyglot' ( http://blog.heroku.com/archives/2011/8/3/polyglot_platform/) programming projects, I would be very interested! I should post that as a question.

如果有人知道一个很好的语言中立、操作系统中立的解决方案，它非常适合“多语言”（http://blog.heroku.com/archives/2011/8/3/polyglot_platform/）编程项目，我会很感兴趣的！我应该把它作为一个问题发布。

回答by MarcH

I think you can manage consistency like this: define a "reference" branch or series of tag with the same name(s) across all your "Core" libraries (note: there is only one "Core" library in your example). Then instruct developers of sub-projects (A, B,...) to regularly upgrade to the reference version of "Core" as soon as they can.

我认为您可以像这样管理一致性：在所有“核心”库中定义一个“参考”分支或一系列具有相同名称的标签（注意：您的示例中只有一个“核心”库）。然后指导子项目（A，B，...）的开发者尽快定期升级到“Core”的参考版本。

Before running a build, easily check that "Core(s)" is consistently used across A, B, C,... by running these three commands in a clean, recursive, "Super", top-level checkout:

在运行构建之前，通过在干净的、递归的、“超级”的顶级检出中运行这三个命令，轻松检查“Core(s)”是否在 A、B、C 中一致使用：

# 1.  Switch to the reference version (= "Force" consistency where need be)
git submodule foreach --recursive 'git checkout [origin/]reference || true'

# 2a. Show which inconsistencies you just forced; terse output
git status -s; git submodule foreach --recursive git status -s 2>/dev/null

# 2b. Same but verbose output
git submodule; git submodule foreach --recursive git submodule

# 3. Switch back to versions individually defined by sub-projects 
git submodule update --recursive

The "Terse output" command 2a above highlights which sub-project(s) are not using the "reference" version of Core.

上面的“简洁输出”命令 2a 突出显示了哪些子项目没有使用 Core 的“参考”版本。

You can easily extend the approach to show diffs, force upgrades, or do any other thing you like.

您可以轻松扩展该方法以显示差异、强制升级或执行您喜欢的任何其他操作。

回答by Antonin Hildebrand

A small utility task turning shared submodules into clones using hard-links might work.

使用硬链接将共享子模块转换为克隆的小型实用程序任务可能会起作用。

You may read my full solution here: https://stackoverflow.com/a/10265084/84283

你可以在这里阅读我的完整解决方案：https: //stackoverflow.com/a/10265084/84283

回答by aaa90210

I would not try and map a dependency tree with sub-modules - for the reasons you have already discovered.

我不会尝试用子模块映射依赖树 - 出于您已经发现的原因。

Sub-modules track a given revision of a given branch, so they are useful of giving a snapshot of a consistent set of modules.

子模块跟踪给定分支的给定修订版，因此它们可用于提供一组一致模块的快照。

So, if your project required a certain set of versions of different modules to be tracked as one unit, you can group them together as sub-modules. You can then tag different set of modules at different versions, to give a history of the project, where each tag shows what versions of what modules were compatible at a point in time.

因此，如果您的项目需要将一组不同模块的特定版本作为一个单元进行跟踪，您可以将它们组合为子模块。然后，您可以标记不同版本的不同模块集，以提供项目的历史记录，其中每个标记显示在某个时间点兼容哪些模块的版本。

 tags/
     release1/ 
           |-> [email protected]
           |-> [email protected]
           |-> [email protected]
     release2/
           |-> [email protected]
           |-> [email protected]
           |-> [email protected]

At least that how I understand them, although like most things with Git, there is probably a whole lot more to it than that. In terms of managing dependencies, all I can say is find another way, it is not what Git with or without sub-modules was designed for as I understand it.

至少我是这样理解它们的，尽管与 Git 的大多数事情一样，它可能还有很多。在管理依赖项方面，我只能说找到另一种方式，这不是我理解的带有或不带有子模块的 Git 的设计目的。