Htykuut

We just started working on something that is maybe best described as a company-wide "open" source framework. Our main language is C# and we use Jenkins and ProGet to create and share nuget-packages. We started putting everything (and I really mean everything. One module has at least three repositories) in it's own Git-repo.
We thought that would be a good idea as we can version and publish everything separately but it turns out to result in a very annoying workflow if you want to make changes on multiple repositories that have dependencies.

I startet looking around and it seems, that most projects use a more monolitic approach and I think that would probably make our lives here easier too. What I am not so sure about is, how versioning with this approach works.

The CoreFx repository is a nice example of what we are trying to achieve. One repo that results in many separate packages. When I build this locally there is one version number for all packages but when I take a look at the available package versions on nuget.org the versions seem to be per package. For example System.Diagnostics.DiagnosticsSource has Version 4.5.1 but references System.Diagnostics.Debug version 4.3.0.

So this is exactly, what I think would be a good solution for us. One repo, many resulting packages with indepentend version.

Does anyone know how this is achieved with the corefx project or has anyone other suggestions, how that could be done?

Both monorepos and very-many-repos have advantages; one example of advantages with many small repos is that you can git tag an individual package's specific version easily. Doing the same in a monorepo is more awkward sometimes.

But, if what you are releasing is more of a "set of related packages", that's perhaps not a bad idea after all: you can have a single .bat or .ps1 file that builds all the NuGet packages for your repository and sets the version number accordingly. Like this example (Powershell script)

The references/dependencies you are talking about (System.Diagnostics.DiagnosticsSource version 4.5.1 depending on System.Diagnostics.Debug version 4.3.0) becomes the individual <dependency> tags in the .nuspec of the "consuming" package (i.e. System.Diagnostics.DiagnosticsSource in this case.) Here, you can choose two major strategies:

• Always bump all <dependency> versions so that "upgrading one upgrades all" the packages. Sometimes this is what you want, and then you should do that.


• Be more fine-grained: only update the dependency when you actually use the newer/updated functionality somehow. For example, when the consuming package uses a method that only exist in a particular version of the dependency.

The latter strategy is arguably the more elegant approach, but also more complex (might require more work to get it done in a good way)

.NET-specific challenges

Assume that I have project A and project B, which has a reference to A and that both are in the same repository. Also I have a Visual Studio solution for project B, which holds a project reference to project A.

Now, I make a change to project B and increment it's version, without incrementing the version of project A. I push my changes and Jenkins makes a clean checkout of the repo and build the solution creating both packages but package A now has the same version as it had on a previous build and can not be pushed to the nuget feed

This is indeed an interesting problem. Some potential workarounds:

• Let Jenkins detect/determine the delta between the changes and only build packages for the parts of the tree that has changed. Probably doable, but perhaps not very easy; requires some logic in your .ps1 script. Perhaps the most elegant solution I can think of.




• Make NuGet pushes that fail because of "version already exists" a non-fatal error. I.e. detect when you run nuget push if you get that particular error (very important detail!) and ignore it. You definitely do not want to ignore all errors in nuget push but only these particular ones. Less elegant, but still fully automatic.




• Make releasing packages a semi-automatic instead of a fully automatic process. Set up one or more jobs in Jenkins where you manually "press the button" when a new version of a package is to be released, perhaps with a job argument saying "which package to release" (can be a dropdown in Jenkins so noone has to type the name manually.) Sounds perhaps a bit manual in nature, but this option is the option that gives you as a team the greatest level of control over what packages gets pushed and when.

Further reading

(About monorepos in general, not specifically dealing with the CoreFx case you mention. Can still provide valuable information though.)

• 
  https://github.com/babel/babel/blob/master/doc/design/monorepo.md: Why is Babel a monorepo?


• 
  https://github.com/korfuri/awesome-monorepo: A curated list of awesome Monorepo tools, software and architectures.


• 
  https://github.com/pouchdb/pouchdb/issues/5128: Some of PouchDB's steps towards a monorepo