infinite recursion when using inputs #742

Closed
opened 2021-10-13 00:39:42 +02:00 by nrdxp · 12 comments
nrdxp commented 2021-10-13 00:39:42 +02:00 (Migrated from github.com)

Mimimal reproduction flake:
https://github.com/nrdxp/repro

Infinite recursion occurs the moment one tries to access inputs from "makes.nix" in any fashion. It happens with both stable and unstable interface. To reproduce simply enter the flake's devshell and run m ..

Mimimal reproduction flake: https://github.com/nrdxp/repro Infinite recursion occurs the moment one tries to access `inputs` from "makes.nix" in any fashion. It happens with both stable and unstable interface. To reproduce simply enter the flake's devshell and run `m .`.
kamadorueda commented 2021-10-13 01:43:27 +02:00 (Migrated from github.com)

Interesting, and also seems simple, I'm gonna drop here examples that work and ones that do not work so you see the pattern

Works:

{ fetchNixpkgs, inputs, ... }:
{
  inputs = {
    nixpkgs = import <nixpkgs> { };
  };
  outputs = {
    "/hello" = inputs.nixpkgs.bash;
  };
}

Works:

{ fetchNixpkgs, inputs, ... }:
{
  inputs = {
    nixpkgs = import <nixpkgs> { };
  };
  outputs = {
    "/hello" = builtins.trace inputs inputs.nixpkgs.bash;
  };
}

Works:

{ fetchNixpkgs, inputs, ... }:
{
  inputs = {
    nixpkgs = import <nixpkgs> { };
  };
  outputs = builtins.trace inputs {
    "/hello" = inputs.nixpkgs.bash;
  };
}

Fails:

{ fetchNixpkgs, inputs, ... }:
{
  inputs = builtins.trace inputs {
    nixpkgs = import <nixpkgs> { };
  };
  outputs = {
    "/hello" = inputs.nixpkgs.bash;
  };
}

Fails:

{ fetchNixpkgs, inputs, ... }:
builtins.trace inputs {
  inputs = {
    nixpkgs = import <nixpkgs> { };
  };
  outputs = {
    "/hello" = inputs.nixpkgs.bash;
  };
}

Fails:

{ fetchNixpkgs, inputs, ... }:
{
  inputs = {
    nixpkgs = import <nixpkgs> { };
  };
  outputs = {
    "/hello" = inputs.nixpkgs.bash;
  };
  a = builtins.trace inputs inputs.nixkgs.bash;
}

inputs depends on evaluating the section of the module system that defines the inputs

if for evaluating the inputs section of the module system you somehow need the inputs, it fails.

since a is not part of the module system, then inputs can't be evaluated, and nixpkgs.lib.modules.evalModules tries to eval the inputs while displaying the error message, which depends on evaluating the module in the first place

Interesting, and also seems simple, I'm gonna drop here examples that work and ones that do not work so you see the pattern Works: ```nix { fetchNixpkgs, inputs, ... }: { inputs = { nixpkgs = import <nixpkgs> { }; }; outputs = { "/hello" = inputs.nixpkgs.bash; }; } ``` Works: ```nix { fetchNixpkgs, inputs, ... }: { inputs = { nixpkgs = import <nixpkgs> { }; }; outputs = { "/hello" = builtins.trace inputs inputs.nixpkgs.bash; }; } ``` Works: ```nix { fetchNixpkgs, inputs, ... }: { inputs = { nixpkgs = import <nixpkgs> { }; }; outputs = builtins.trace inputs { "/hello" = inputs.nixpkgs.bash; }; } ``` Fails: ```nix { fetchNixpkgs, inputs, ... }: { inputs = builtins.trace inputs { nixpkgs = import <nixpkgs> { }; }; outputs = { "/hello" = inputs.nixpkgs.bash; }; } ``` Fails: ```nix { fetchNixpkgs, inputs, ... }: builtins.trace inputs { inputs = { nixpkgs = import <nixpkgs> { }; }; outputs = { "/hello" = inputs.nixpkgs.bash; }; } ``` Fails: ```nix { fetchNixpkgs, inputs, ... }: { inputs = { nixpkgs = import <nixpkgs> { }; }; outputs = { "/hello" = inputs.nixpkgs.bash; }; a = builtins.trace inputs inputs.nixkgs.bash; } ``` inputs depends on evaluating the section of the module system that defines the inputs if for evaluating the inputs section of the module system you somehow need the inputs, it fails. since `a` is not part of the module system, then inputs can't be evaluated, and nixpkgs.lib.modules.evalModules tries to eval the inputs while displaying the error message, which depends on evaluating the module in the first place
nrdxp commented 2021-10-13 01:58:49 +02:00 (Migrated from github.com)

I see, what we really wanted to do was pass a reference to the self input to the nomad deployment task like so:

{ inputs, ... }:
{
  nomadDeployNamespaces = inputs.self.workloads;
}
I see, what we really wanted to do was pass a reference to the `self` input to the nomad deployment task like so: ```nix { inputs, ... }: { nomadDeployNamespaces = inputs.self.workloads; } ```
kamadorueda commented 2021-10-13 01:58:52 +02:00 (Migrated from github.com)

there is not much we can do about the specific case you mention:

{ inputs, ... }:
{
  foo = inputs;
}

it fails even if you did not use inputs at all:

{ inputs, ... }:
{
  foo = 123;
}
# The option `foo' does not exist. Definition values:
# - In `<unknown-file>': 123

the infinite recursion is born after a failure in the first place,
so just remove the original failure (option foo does not exist) and it'll work as expected

there is not much we can do about the specific case you mention: ```nix { inputs, ... }: { foo = inputs; } ``` it fails even if you did not use inputs at all: ``` { inputs, ... }: { foo = 123; } # The option `foo' does not exist. Definition values: # - In `<unknown-file>': 123 ``` the infinite recursion is born after a failure in the first place, so just remove the original failure (option foo does not exist) and it'll work as expected
kamadorueda commented 2021-10-13 02:03:34 +02:00 (Migrated from github.com)

I see, what we really wanted to do was pass a reference to the self input to the nomad deployment task like so:

{ inputs, ... }:
{
  nomadDeployNamespaces = inputs.self.workloads;
}

Just a few ideas on where the failure may be:

  • Have you checked nomadDeployNamespaces is an option?
  • Have you checked that are you evaluating your project with a Makes src/commit that offers nomadDeployNamespaces ?

normally for testing I just replace makes-src with /path/to/my/local/dev/makes

> I see, what we really wanted to do was pass a reference to the `self` input to the nomad deployment task like so: > > ```nix > { inputs, ... }: > { > nomadDeployNamespaces = inputs.self.workloads; > } > ``` Just a few ideas on where the failure may be: - Have you checked `nomadDeployNamespaces` is an option? - Have you checked that are you evaluating your project with a Makes src/commit that offers `nomadDeployNamespaces` ? normally for testing I just replace makes-src with /path/to/my/local/dev/makes
blaggacao commented 2021-10-13 02:06:59 +02:00 (Migrated from github.com)

This is probably a (camouflaged) variant of https://github.com/fluidattacks/makes/pull/575

This is probably a (camouflaged) variant of https://github.com/fluidattacks/makes/pull/575
blaggacao commented 2021-10-13 02:10:04 +02:00 (Migrated from github.com)

the infinite recursion is born after a failure in the first place

Ah! Very interesting 🛩️

> the infinite recursion is born after a failure in the first place Ah! Very interesting :small_airplane:
nrdxp commented 2021-10-13 02:10:37 +02:00 (Migrated from github.com)

So nomadDeployNamespaces should exist and I am pulling the makes script directly from our WIP branch which contains all the needed bits from the Nomad PR as rebased on some of our own patches. So unless there is some typo somewhere, these the conditions for success should be met in this case, unless I am misunderstanding the situation?

So `nomadDeployNamespaces` should exist and I am pulling the makes script directly from [our WIP branch](https://github.com/input-output-hk/makes/tree/WIP-nomad) which contains all the needed bits from the Nomad PR as rebased on some of our own patches. So unless there is some typo somewhere, these the conditions for success should be met in this case, unless I am misunderstanding the situation?
kamadorueda commented 2021-10-13 02:45:44 +02:00 (Migrated from github.com)

I just checked-in your branch and:

image

From what I see you added a module here: github.com/input-output-hk/makes@d834344bd4

and it only defines:

{
  options = {
    deployNomad = {
      modules = lib.mkOption {
        default = { };
        type = lib.types.attrsOf (lib.types.submodule (_: {
          options = {
# ...
            namespace = lib.mkOption {
              type = lib.types.str;
            };
# ...
        }));
      };
    };
}

So you should not be configuring it like: nomadDeployNamespaces = xxx

but more like this:

# makes.nix
{
  deployNomad.modules = {
    example = {
      namespace = "xxx";
      # ...
    };
  };
}

and then m . /deployNomad/example

I just checked-in your branch and: ![image](https://user-images.githubusercontent.com/47480384/137047391-422ce3d9-9b05-437b-8676-627fa18c8ea9.png) From what I see you added a module here: https://github.com/input-output-hk/makes/commit/d834344bd463b7684e89c73346862732f57a5b41 and it only defines: ```nix { options = { deployNomad = { modules = lib.mkOption { default = { }; type = lib.types.attrsOf (lib.types.submodule (_: { options = { # ... namespace = lib.mkOption { type = lib.types.str; }; # ... })); }; }; } ``` So you should not be configuring it like: `nomadDeployNamespaces = xxx` but more like this: ```nix # makes.nix { deployNomad.modules = { example = { namespace = "xxx"; # ... }; }; } ``` and then `m . /deployNomad/example`
kamadorueda commented 2021-10-13 02:49:13 +02:00 (Migrated from github.com)
  1. nix-env -i just
  2. Inside your makes repo: $ just m . /deployNomad/example (this builds a dev m and runs the thing)
diff --git a/makes.nix b/makes.nix
index 47677de..8ea678e 100644
--- a/makes.nix
+++ b/makes.nix
@@ -13,6 +13,13 @@
       pubKey = "makes.cachix.org-1:HbCQcdlYyT/mYuOx6rlgkNkonTGUjVr3D+YpuGRmO+Y=";
     };
   };
+  deployNomad.modules = {
+    example = {
+      namespace = "xxx";
+      version = "1.0";
+      src = "/tests";
+    };
+  };
   deployContainerImage = {
     images = {
       makesGitHub = {

image

it now works! (fails because I used a fake directory for example purposes)

1. nix-env -i just 1. Inside your makes repo: `$ just m . /deployNomad/example` (this builds a dev `m` and runs the thing) ```diff diff --git a/makes.nix b/makes.nix index 47677de..8ea678e 100644 --- a/makes.nix +++ b/makes.nix @@ -13,6 +13,13 @@ pubKey = "makes.cachix.org-1:HbCQcdlYyT/mYuOx6rlgkNkonTGUjVr3D+YpuGRmO+Y="; }; }; + deployNomad.modules = { + example = { + namespace = "xxx"; + version = "1.0"; + src = "/tests"; + }; + }; deployContainerImage = { images = { makesGitHub = { ``` ![image](https://user-images.githubusercontent.com/47480384/137048115-3ff71ba8-2e72-441e-95c2-259ca81d30f5.png) it now works! (fails because I used a fake directory for example purposes)
blaggacao commented 2021-10-13 13:40:46 +02:00 (Migrated from github.com)

Ha! Who had thought a forgotten rebase of mine / wrong commit would lead to an inifinte recursion error. 😄

@kamadorueda thanks for the excellent hunting down!

Ha! Who had thought a forgotten rebase of mine / wrong commit would lead to an inifinte recursion error. :smile: @kamadorueda thanks for the excellent hunting down!
nrdxp commented 2021-10-13 17:22:02 +02:00 (Migrated from github.com)

sorry for false alarm, and thanks for the help! 🙏

sorry for false alarm, and thanks for the help! :pray:
kamadorueda commented 2021-10-15 06:43:40 +02:00 (Migrated from github.com)

This kind of things are gonna help with such debugging in future situations:

https://github.com/NixOS/nix/pull/5389

This kind of things are gonna help with such debugging in future situations: https://github.com/NixOS/nix/pull/5389
This discussion has been locked. Commenting is limited to contributors.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
justin/makes#742
No description provided.