Support Nix Flakes #546

Open
opened 2021-09-16 21:23:28 +02:00 by kamadorueda · 2 comments
kamadorueda commented 2021-09-16 21:23:28 +02:00 (Migrated from github.com)

Related:

Useful links:

Examples:

Things to take into account:

  • We wan to benefit from flakes hermeticity
  • We want to keep installation and usage of non-nix-experts users as simple as it's today
Related: - #512 - #540 Useful links: - https://nixos.wiki/wiki/Flakes Examples: - https://github.com/numtide/flake-utils/blob/master/flake.nix Things to take into account: - We wan to benefit from flakes hermeticity - We want to keep installation and usage of non-nix-experts users as simple as it's today
kamadorueda commented 2021-09-17 21:41:10 +02:00 (Migrated from github.com)

I'm just gonna drop here two good reasons not to use flakes yet:


  1. It's unstable, which means bugs in the core binary do exist and will sooner or later bite you, no matter what --option xxx yyy, ulimit, etc, you configure to try controlling file descriptor leaks:

image

for some companies like us it's not reasonable to risk reliability on production, the thing must just work, preferably for ever

on Nix stable this too-many-open-files do not happen (downgrading from Flakes was actually the fix to the problem)


  1. Evaluation cache is awesome on flakes, but runtime cache-ability is not perfect yet, for instance we cannot make use of projectPath or something similar:

image

which means that in a tree like following:

  • a
    • whatever-code
  • b
    • c.sh
    • d.nix (depends on c.sh)

if whatever-code changes, then the entire inputs.self.sourceInfo.outPath changes, and thus ./b/c.sh changes, and ./b/d.nix must be build again. This does not happen on Makes using Nix stable because using projectPath you can make d.nix depend only on the NAR serialization of b folder, and changes outside of b do not require to rebuild d.nix.

Maybe this is the reason why some nix experts are using Bazel to achieve incremental builds?

We achieved incremental builds at Fluid Attacks using nix stable and projectPath only


I'm gonna stop working towards Flakes for now

The work done so far serves as a draft on how compatibility with Flakes can be implemented

I'm just gonna drop here two good reasons not to use flakes yet: --- 1. It's unstable, which means bugs in the core binary **do exist** and will sooner or later bite you, no matter what `--option xxx yyy`, `ulimit`, etc, you configure to try controlling file descriptor leaks: ![image](https://user-images.githubusercontent.com/47480384/133842604-ecca7596-c3f4-4e27-aa6c-aa1e047fe8e3.png) for some companies like us it's not reasonable to risk reliability on production, the thing must just work, preferably for ever on Nix stable this `too-many-open-files` do not happen (downgrading from Flakes was actually the fix to the problem) --- 2. Evaluation cache is awesome on flakes, but runtime cache-ability is not perfect yet, for instance we cannot make use of `projectPath` or something similar: ![image](https://user-images.githubusercontent.com/47480384/133843180-80364346-4253-4e15-a1e2-c95bfa3d15f5.png) which means that in a tree like following: - a - whatever-code - b - c.sh - d.nix (depends on c.sh) if `whatever-code` changes, then **the entire** `inputs.self.sourceInfo.outPath` changes, and thus `./b/c.sh` changes, and `./b/d.nix` must be build again. This does not happen on Makes using Nix stable because using projectPath you can make `d.nix` depend only on the NAR serialization of `b` folder, and changes outside of `b` do not require to rebuild `d.nix`. Maybe this is the reason why some nix experts are using Bazel to achieve incremental builds? We achieved incremental builds at Fluid Attacks using nix stable and `projectPath` only --- I'm gonna stop working towards Flakes for now The work done so far serves as a draft on how compatibility with Flakes can be implemented
Ten0 commented 2023-12-24 09:22:24 +01:00 (Migrated from github.com)

2. runtime cache-ability is not perfect yet, for instance we cannot make use of projectPath or something similar:

Unless I'm mistaken this is fixed the next day by github.com/fluidattacks/makes@2143be2f8f, correct?

> 2\. runtime cache-ability is not perfect yet, for instance we cannot make use of `projectPath` or something similar: Unless I'm mistaken this is fixed the next day by https://github.com/fluidattacks/makes/commit/2143be2f8f00ac6ef530d029e72dc71f59d7a001, correct?
This discussion has been locked. Commenting is limited to contributors.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
justin/makes#546
No description provided.