Allow users to customize outputs list #1223
Labels
No labels
dependencies
difficulty::high
difficulty::low
difficulty::medium
good first issue
help wanted
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
justin/makes#1223
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I'm considering using this project.
IIUC, I currently have two ways to generate an "output" for the CLI:
builtin, and that will generate an arbirary number of targets depending on some magic inmakes' own codeextension, which always means creating a folder for the target, and having amain.nixin that folder.What I want to achieve is the following directory structure:
and I want my possible "OUTPUT"s in the CLI to look like this:
where the build/test/run/deploy outputs behavior are all defined by my
/makes/service.nixThat means I need to be able to customize:
<main or makes>.nixs (so that with a single nix file in a service folder I can declare the multiple targets that I need)<main or makes>.nixs (to be able to passimport ./makes/service.nix)My understanding is that this is not possible currently, and instead any such
service.nixwould have to be declared as a builtin inmakesitself, and then I would put only the configuration of that in somemakes.nix, which also means I could only instantiate one of each of those permakes.nix, whereas my ideal syntax would probably be:(where
outputAttrSetwould convert from the attrset of output to probably[{ name, derivation }], allowing for further customization if required)Is it actually already possible to achieve something like this? If so, how? If not, is this something that could maybe be made possible in the future or are there any clear blockers?
Thanks,
Hey @Ten0,
For
main.nixfiles, you can setextendingMakesDirs = ["/"];as described in the documentation. With this setting Makes will name CLI jobs according to absolute paths within the repository. That is, for file/project1/service1/run/main.nix, the CLI output would be/project1/service1/run.With
makes.nixthings are a little more complicated.makes.nixis an interface between:makes.nix.This is why, when declaring builtins from
makes.nixfiles, CLI outputs do not follow a naming pattern based on directory structure likemain.nixfiles but rather group all jobs for the same builtin.Example:
For the following
makes.nixfileWe get the following CLI outputs:
With some boilerplate code you could still use all builtins from
main.nixfiles, granting directory-based CLI outputs.Example:
/project1/service1/format/main.nixFrom that
main.nixyou can do anything that nix allows so you can orchestrate everything from amakes/services.nixfile:/project1/service1/format/main.nixYou could even declare everything in the
services.nixfile and simply import it from themain.nix:Thank you for your answer! 🙂
So it looks like my understanding was correct: to generate target paths, I'm tied by either the builtins (which I can't customize) or the "one folder with a main.nix per output" approach (which would force me to create one directory per output with a main file that would specify a bunch of boilerplate and finally
.target_name(=folder_name), which woudn't fit at all with my current directory structure or be one I'd like anyway) 🙁Is there any technical constraint that prevents building a system that has a more flexible interface similar to the one I described above? Having the root expression that evaluates to just at-most-one-incantation-of-each-type-of-builtin doesn't allow for flexible program structure compared to something along the lines of
[{ name, thing_to_do }].At first glance it looks like we could evaluate only the part of the current folder's expressions that are related to outputs names when starting the CLI, and then only when actually building, evaluate the
derivationattribute of each{ name, derivation }that the root expression outputs, but there may be big drawbacks that I'm missing.Would a design where we add an
outputsbuiltin that takes as parameter[{ name, derivation }]work or is there some constraint that prevents passing such complex inputs to builtins? (like, they have to get evaluated before they get passed or something along those lines)To be very clear: I love the idea of the project of having a CI system globally powered by nix, with a CLI that enables choosing the targets we're interested in, especially making full use of caching across all users of the organization for every single derivation. The potential of it looks insane! However, if it seems that the only way to efficiently add outputs to the CLI is to make incantations in the
makes.nixafter PR-ing a new builtin to this repository, then it seems pretty clear to me that that doesn't scale unless you happen to also be the company that manages theMakesproject: we can't have every company use this and PR every single customization they need as a global builtin in the project:Thanks a lot 🙂
I will start working on a first iteration that allows to declare
jobs = {}withinmakes.nixso we can begin moving away frommain.nixfiles.@Ten0 Hey! If you're still interested, take a look at the new
jobs = {}essential. It basically allows to describe jobs directly inmakes.nixfiles. https://makes.fluidattacks.tech/configuration/essentials/#jobsHey! Thanks for letting me know! That's indeed very interesting!
I may give this another look in the coming weeks. 👍️
main.nixfiles in Universemain.nixsupport from Makes CLImain.nixsupport from Makes backend