Advice on how to import private sources (makes breaks builtins.fetchGit?) #941

Closed
opened 2022-09-28 13:44:17 +02:00 by davidreuss · 2 comments
davidreuss commented 2022-09-28 13:44:17 +02:00 (Migrated from github.com)

Hello,

Currently evaluating makes for replacing a bunch of bespoke tooling for unifying developer laptop setups and CI.

So far really happy with usage of makes, and it's a really good project... but;

I'm a bit worried about all the wrangling that's going on underneath the hood. I'm a newcomer to nix, but ... i'm stumped by the below problem currently.

We have some private repositories with expressions that i would like to reuse with makes. Dependency management is done with niv much like you're using it in makes itself.

So i'm currently getting this error;

error: cannot run ssh: No such file or directory
fatal: unable to fork
error: program 'git' failed with exit code 128

       … while fetching the input 'git+ssh://git@github.com/my/private-repo-nix?ref=foobar'

       … while evaluating 'makeSource'

       at /nix/store/sm9k5pr0j9rxgafw9h3fcbkc2nzdwmri-src/src/args/make-search-paths/default.nix:42:16:

My module/extension look like:

❯ cat makes/custom/main.nix
{ inputs
, makeScript
, ...
}:

let
  mypkgs = builtins.fetchGit {
    url = "git@github.com:my/private-repo-nix";
    ref = "foobar";
  };
in

makeScript {
  name = "example";
  searchPaths.source = [
    mypkgs
  ];
  entrypoint = "echo Hello world";
}

And, if i evaluate the builtins.fetchGit {} expression via nix repl, or do a plain nix-build it works just fine...

I hacked around trying to pick apart how nix is getting invoked ... and using that specific nix from the store path, i still don't see the breakage evaluating the basic expression.

Can you help me shed some light on this issue? I would really want to get to a place where i'd recommend makes, but this seemingly simple problem has caught me a little by surprise.

The problem seems to be both present with makes v22-10 and makes v22-09.

Thanks in advance 🙏

Hello, Currently evaluating makes for replacing a bunch of bespoke tooling for unifying developer laptop setups and CI. So far really happy with usage of `makes`, and it's a really good project... but; I'm a bit worried about all the wrangling that's going on underneath the hood. I'm a newcomer to `nix`, but ... i'm stumped by the below problem currently. We have some private repositories with expressions that i would like to reuse with `makes`. Dependency management is done with `niv` much like you're using it in `makes` [itself](https://github.com/fluidattacks/makes/blob/main/src/nix/sources.json). So i'm currently getting this error; ``` error: cannot run ssh: No such file or directory fatal: unable to fork error: program 'git' failed with exit code 128 … while fetching the input 'git+ssh://git@github.com/my/private-repo-nix?ref=foobar' … while evaluating 'makeSource' at /nix/store/sm9k5pr0j9rxgafw9h3fcbkc2nzdwmri-src/src/args/make-search-paths/default.nix:42:16: ``` My module/extension look like: ``` ❯ cat makes/custom/main.nix { inputs , makeScript , ... }: let mypkgs = builtins.fetchGit { url = "git@github.com:my/private-repo-nix"; ref = "foobar"; }; in makeScript { name = "example"; searchPaths.source = [ mypkgs ]; entrypoint = "echo Hello world"; } ``` And, if i evaluate the `builtins.fetchGit {}` expression via `nix repl`, or do a plain `nix-build` it works just fine... I hacked around trying to pick apart how nix is getting invoked ... and using that specific nix from the store path, i still don't see the breakage evaluating the basic expression. Can you help me shed some light on this issue? I would really want to get to a place where i'd recommend `makes`, but this seemingly simple problem has caught me a little by surprise. The problem seems to be both present with `makes v22-10` and `makes v22-09`. Thanks in advance 🙏
kamadorueda commented 2022-09-28 17:25:27 +02:00 (Migrated from github.com)

Good catch, in general, this is related to: https://github.com/NixOS/nix/issues/3533.

Nix doesn't bundle git nor ssh, or other binaries it needs. However, we currently bundle git with the makes CLI, so builtins.fetchGit is able to find git in the PATH even if the user has not installed it in the host, but, we didn't wrap ssh, but this is easy to fix: https://github.com/fluidattacks/makes/pull/942

@davidreuss Could you please help me test if it works now? Since this is a change in the CLI, you would need to install the latest version of the CLI, like: nix-env -if https://github.com/fluidattacks/makes/archive/main.tar.gz or something similar. You may need to $ ssh-add your private ssh key before running makes. I tested it on my machine and it seems to work, but since this is an impurity of Nix I'm not sure if it would work on every machine out there. Thanks for your help testing this!

Good catch, in general, this is related to: <https://github.com/NixOS/nix/issues/3533>. Nix doesn't bundle git nor ssh, or other binaries it needs. However, we currently bundle git with the makes CLI, so builtins.fetchGit is able to find git in the PATH even if the user has not installed it in the host, but, we didn't wrap ssh, but this is easy to fix: https://github.com/fluidattacks/makes/pull/942 @davidreuss Could you please help me test if it works now? Since this is a change in the CLI, you would need to install the latest version of the CLI, like: `nix-env -if https://github.com/fluidattacks/makes/archive/main.tar.gz` or something similar. You may need to `$ ssh-add` your private ssh key before running makes. I tested it on my machine and it seems to work, but since this is an impurity of Nix I'm not sure if it would work on every machine out there. Thanks for your help testing this!
davidreuss commented 2022-09-28 18:04:39 +02:00 (Migrated from github.com)

That was fast.

I tried adding openssh package in src/args/make-derivation/default.nix and following the trail .. but seems like i never found the exact one to actually fix 😅

Confirmed that it works just as expected. 🎉

Thank you for the quick fix 🥳

That was fast. I tried adding `openssh` package in `src/args/make-derivation/default.nix` and following the trail .. but seems like i never found the exact one to actually fix 😅 Confirmed that it works just as expected. 🎉 Thank you for the quick fix 🥳
This discussion has been locked. Commenting is limited to contributors.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
justin/makes#941
No description provided.