How do I organize or manage n number of submodules across m number of projects?

Question

I'm working on a project and no one can agree on the best way to solve this. Basically the project uses git submodules to bring in several "library" packages. It's an obscure language/framework, no package manager ecosystem. I had nothing to do with any of this, I just happened to take over maintenance along with a few other developers of various levels of expertise.

So we have apps:

• Foo
• Bar
• Baz
• Fip
• Zoo

And libraries:

• LibZip
• LibZap
• LibGup
• LibGap
• LibRuf
• LibRug
• LibRuk
• LibRut

Each app only has one main branch and most of the apps use all of the libraries, with a few exceptions, such as Zoo only uses LibZip and LibRuf.

However, Foo and Bar, for example, use different versions of LibZip. So LibZip has different branches main, foo, and bar. No one can agree how to organize LibZip. Some want to use branch names, tags, separate fork repos. Some want to create new forks for each version of the app. Others want a single branch history with feature toggles (which I personally loathe).

My team happens to have one person per app, but the libraries have been the source of some friction/disagreement regarding the libraries. Has anyone here worked in a situation like this? Or does anyone know of an example I can point the team at and say "Oh, take a look at this team repo for NotActuallyWordPress and how they did it."

Answer 1

I think one good thing to do would be to refactor the code to use the same versions of libraries in all applications, if the applications belong to the same project. Perhaps build a meta package which has all the dependencies, and point to that.

If they don't belong to the same project, I think the best thing would be to create a diagram and find the cleanest (most linear, IMHO) solution.

If you want successful examples, take a look at what GNOME and KDE did.

Answer 2

The only project that I can remember using submodules was Symfony. It's a PHP framework structured in so-called "bundles". You have a folder in your project's tree called vendor/ inside where you'd pull all the bundles that you need for your application as git submodules, and the framework would autoload these modules.
However if you're using submodules, I would assume that it is because you need to customize them or you need to contribute code to the various modules, since you're pulling all their trees. Otherwise if it's just a way for sharing code, I would say that it's simpler to remove all the submodules and distribute the code as a shared/static library or a module (depending on the language).
Anyway, a submodule is just a reference to another repository at a specific commit. When you add a submodule, git will create a link to the top commit. You can enter the folder where the submodule has been saved and checkout a specific branch, tag, or commit hash. For example you can cd in the submodule and git checkout v2.0. Since this link has now changed, you will need to commit this change in your project for example git commit -m "Upgrade dependency to v2.0". When someone else clones your repository (and the submodules recursively), your repository will maintain the link to the external repository at that specific commit.
The difference between "branch" vs "tag" matters from the point of view of your library, LibZip in the example, not much from the point of view of the submodule. The difference between tags and branches is that branches always points at the top of a development line and this reference changes at every commit, whereas a tag is a reference to a specific commit and won't change.
The way that LibZip is structured in branches/tags should be independent of how it's used in submodules. Use branches or tags in LibZip as it makes sense for it. Then when another repository includes it as a submodule, checkout a specific tag or branch in the submodule, inside your repo. "New forks" means that you have to manage more submodules, instead of 1 submodule where you checkout at different commits. "Single branch with different toggles" could make sense for example if you can compile a file with various options. In this case you'd add the submodule and when you compile your project you turn features on/off for your specific app (for example --language=ru). Branches, tags, feature toggles have their place and you should use these as it makes the most sense for the repository itself, not for other repositories.