Zee3 - Bindings to the Z3 theorem prover

I have written some bindings to the Z3 theorem prover. The Z3 executable is used as part of a port (i.e. I am not binding to the executable using the C API), which allows for interactive proofs using the Zee3.push(solver_pid) and Zee3.pop(solver_pid) functions. The bindings are heavily inspired by the Python bindings (link here), but with some Elixir twists: the main interaction happens inside a macro which allows us to pretend we are not interacting with a Port, while keeping everything outside of the macro very explicit. Technical note: the Z3 executable is downloaded during compilation of the package, but so far I have only tested it on Linux (Ubuntu 22.4). Downloading for other platforms may fail. You can always download the package for your system and set it as the executable using the config :zee3, z3_executable: "z3"

Hex package here and github repo here.

For those who don’t know what Z3 is, it is simply a magical (I don’t use the term lightly) program which allows you to define constants and constraints satisfied by those constants and will tell you whether the constraints are satisfiable or not. By encoding a property of the system (a program, a scheduling program, etc.) as a satisfiability problem, Z3 will tell you whether it’s true or not, and which assignment of constants will make it true. It does so by extending SAT solving with more complex theories pertaining to more complex objects such as Integers, Real numbers, Floating Point numbers (which respect the rules of floating point numbers instead of actual real numbers), Bit Vectors and Regular expressions.

It has some common “sub-engines” as a special case, such as a Simplex solver (which does the same as my Dantzig package as a special case) and a Datalog engine. I can seamlesly combine, for example, the Simplex solver with some more general constraints (which will obviously destroy the fast runtime of the simplex solver).

You can use this to find an input of any function f(x1, x2, ...) = y given y and the implementation for f. Note that Z3 may fail to find whether a formula is satisfiable (in this case, it returns unknown) or may take time which is exponential as a function of the length of the size of the problem.

The package still needs proper documentation, but I don’t antecipate anything to change in the public-facing API. Contributions are very welcome.

Examples

The best way to get a feeling for how it works, is to look at some of the tests.

Example 1

The comments explain the details of the code, but the basic idea is the following:

  1. We start the solver as a stateful inteval process with which we will communicate through message passing
  2. We create a Zee3.program which allows us to add the declarations and the constraints, while seamlessly mutating the internal state of the Zee3 solver to add the appropriate variables
  3. We check_and_get_model!() to check for satisfiability and get the model (i.e. a set of variable assignments that satisfies the constraints)
  4. We return the last expression in the program, which happens to be the value for check_and_get_model!(), but it doesn’t need to be, like the next example will show
  test "simple program" do
    {:ok, pid} = Zee3.start_solver()

    zero = 0

    {:sat, model} =
      Zee3.program pid do
        # Declare integer constants
        # Note: there is nothing special with the `Sort.int()` function call,
        # the `Sort` module is just the `Zee3.StdLib.Sort` module, which the
        # `Zee3.program` macro aliases inside the body so we can refer to it
        # without needing to alias it ourselves.
        a = declare_const("a", Sort.int())
        b = declare_const("b", Sort.int())

        # Declare a function that takes two integers and returns an integer
        f = declare_fun("f", [Sort.int(), Sort.int()], Sort.int())

        # Assert a constraint (note that Zee3 recognizes literal integers correctly)
        assert f.(a, b) == -5
        # Assert a new constraint, using a variable defined outside of the program
        assert f.(a, b) + f.(b, a) == zero
        # Note: the `assert/2` above is a Z3 assertion, not an ExUnit assertion.

        # Check for satisfiability and get the model if satisfiable
        check_sat_and_get_model!()
      end

    # "normal" assert, taken from ExUnit and unrelated to Z3
    assert model["f"].(model["b"], model["a"]) == 5
    # Note that the `model["f"]` is an actual anonymous function
    # which can be called from Elixir like any other function
    # (of course this function only takes "intersting" values
    # in the values we assert in the program)
  end

Example 2

What if we need to return two values from the program? That is absolutely not a problem, because internally the program retains the semantics of variable assignments so that we can assiign our results to variables and return them ast the end.

  test "forgets constraints and variables whe popped", %{solver: solver} do
    # We want to check for the satisfiability of two different
    # constraint systems using the `push()` and `pop()` functions
    # to add and remove new constraints.
    #
    # Because we have two different satisfiability results,
    # we need to return the result of *two different*
    # `check_sat_and_get_model!()`.
    #
    # This is absolutely not a problem, because the `Zee3.program`
    # respects Elixir's semantics around variable assignment and
    # we can simply assign the results to variables, or save them
    # in a map, or whatevet, and simply return the results at the end.
    {result_1, result_2} =
      Zee3.program solver do
        a = declare_const("a", Sort.int())
        assert a > 0
        # Create a new context here
        push()
        temp_b = declare_const("temp_b", Sort.int())
        assert temp_b == a
        assert temp_b < 0

        # This variable is not made available outside the program,
        # we will need to return it as part of the last expression.
        result_1 = check_sat_and_get_model!()

        # Pop the context that defined the `temp_b` variable
        pop()

        # As above, the variable is not made available outside
        # the program
        result_2 = check_sat_and_get_model!()

        # Return the two results
        {result_1, result_2}
      end

    # Refers to the new context
    assert :unsat == result_1
    # Refers to the original context
    assert {:sat, model} = result_2
    assert model["a"] > 0
    refute Map.has_key?(model, "temp_b")
  end

Example 3

Define variables dynamically using “normal” elixir constructs:

  test "for loops work inside the program", %{solver: solver} do
    {:sat, model} =
      Zee3.program solver do
        # Dynacmially create a number of variables and store them
        # in a list. In this case, we create 10 variables, with
        # names of the form x_i (for i in 1..10).
        xs =
          for i <- 1..10 do
            # Note that there is nothing special about the first
            # argument of `declare_const/2`, which can be anything
            # that returns a string.
            _x_i = declare_const("x_#{i}", Sort.int())
          end

        # Assert that they sum to 10.
        # Where does the `sum/1` function come from?
        # It's just a function that the `Zee3.StdLib` module defines
        # and imports inside the program. You can easily define your
        # own functions and use them inside the program, as long
        # as the functions return the right format, as documented
        # elsewhere
        assert sum(xs) == 10

        # Assert pairwise comparisons between all the variables.
        # Note: there is actually a built in for this, but we really
        # wanted to show that we can use for loops and normal Elixir
        # functions without any issues
        for {x_i, x_i_plus_1} <- Enum.zip(Enum.drop(xs, 1), xs) do
          assert x_i == x_i_plus_1
        end

        check_sat_and_get_model!()
      end

    # Asert that all variables exist and are set to the only
    # value that satisfies the given constraints
    assert model["x_1"] == 1
    assert model["x_2"] == 1
    assert model["x_3"] == 1
    assert model["x_4"] == 1
    assert model["x_5"] == 1
    assert model["x_6"] == 1
    assert model["x_7"] == 1
    assert model["x_8"] == 1
    assert model["x_9"] == 1
    assert model["x_10"] == 1
  end

https://github.com/tmbb/zee3

5 Likes

That sounds intriguing; I use z3 in Cure ( Type System — Dependently-Typed BEAM Language ) and I might maybe benefit from using this library instead of shelling out.

I didn’t know anyone here was using Z3 for non-trivial tasks, it’s great to see that other people see the use of Z3 (and see the use of avoiding Python when possible).

If you are using Z3 as an interactive theorem prover (i.e. using (push) and (pop)) I think you can benefit. If you run a new instance of Z3 everytime you want to check something, I’m not sure what the real benefit is. I am currently adding an escape hatch to add literal SMT-LIB2 text to the running Z3 process (you can already do that using Port.command, but it’s not part of the public API). Feel free to contribute if you find this useful!

1 Like

I have just added some new functionality. One can now define functions inside an elixir module, import them into Z3 inside the Zee3.program and use those functions inside the program. These are “real function calls” and not merely the result of expanding the function in Elixir. Here is an example from the test suite:

defmodule Zee3.ProgramTest do
  use ExUnit.Case, async: true
  require Zee3
  alias Zee3.Smt2

  # ... omitted setup stuff

  defmodule AltLib do
    # Hide the Kernel operators and import the Z3 builtins
    use Zee3.StdLib
    # Make the `defzee3` macro available, as well as performing
    # some behind the scenes magic which will allow the program
    # to use this module correctly
    use Zee3.Defzee3

    @doc """
    Defines a Z3 function, which can be invoked if the module
    is used inside a program.
    """
    defzee3 funky_addition(a :: Sort.int(), b :: Sort.int()) :: Sort.int() do
      # Note that this is using the builtin Z3 operators
      # instead of the default Kernel operators
      a * b - 1
    end
  end

  test "functions defined with defzee3 work", %{solver: solver} do
    {{:sat, _model}, func_call} =
      Zee3.program solver do
        # TODO: find a better API than using the module
        use Zee3.ProgramTest.AltLib, with_alias: AltLib

        x = declare_const("x", Sort.int())
        y = declare_const("y", Sort.int())

        # When is funky addition equal to normal addition?
        assert AltLib.funky_addition(x, y) == x + y

        # Now, let's make sure `AltLib.funky_addition/2`
        # returns a Z3 function call instead of expanding
        # the function body itself.
        # To do this, we will call the function and assign
        # the result to a variable and outside the `Zee3.program`
        # we will serialize the contents of the variable.
        func_call = AltLib.funky_addition(x, y)

        {check_sat_and_get_model!(), func_call}
      end

    # The function call returns the internal representation
    # of a function call instead of expanding the
    assert Smt2.serialize(func_call) == "(Zee3.ProgramTest.AltLib.funky_addition x y)"
  end
end

The syntax to import the module is use Full.Path.To.MyModule, with_alias: MyModule (the alias is not optional). Currently the use ... macro is processed in a special way inside the Zee3.program whenever it has a single with_alias: ... option. This means it is not a “real” use macro. If your use macros don’t take a single with_alias: ... option, they will be processed normally. Note that the name of the function defined inside Z3 is the fullly qualified function name to avoid clashes. This allows you to have two modules with functions with the same name.

Dumb naming question: the idiom use Full.Path.MyModule, with_alias: MyModule is not very explicit regarding the fact that some Zee3 magic is happening. What about use Full.Path.MyModule, zee3_alias: MyModule? now zee3 appears somewhere in the use macro arguments…

Alternatively, I could use the following syntax construction, which is cleaner:

use Zee3.UserLib, module: Full.Path.To.MyModule, alias: MyModule

Some updates on the implementation: I have added some major functionality which will probabbly make this package very useful for other applications.

Datalog implementation

I have added bindings to the Z3 datalog engine (called muZ), which despite not being very performant, it is much more performant than I can implement natively in Elixir, especially when one needs the extensions to standard datalog that Z3 provides.

    require Zee3

    test "simple program with string entities", %{solver: pid} do
      {:ok, {:sat, solutions}} =
        Zee3.program pid do
          # Make sure the relations between entities declare entities
          # with the right sort from the standard library.
          edge = declare_rel("edge", [Sort.entity_id(), Sort.entity_id()])
          path = declare_rel("path", [Sort.entity_id(), Sort.entity_id()])
          # Declare variables, which, again, will use the correct sort.
          a = declare_var("a", Sort.entity_id())
          b = declare_var("b", Sort.entity_id())
          c = declare_var("c", Sort.entity_id())

          # Transitive closure definition
          rule(path.(a, b) <- edge.(a, b))
          rule(path.(a, c) <- path.(a, b) and path.(b, c))

          rule(edge.(entity_id("alice"), entity_id("bob")))
          rule(edge.(entity_id("alice"), entity_id("charlie")))
          rule(edge.(entity_id("bob"), entity_id("dan")))

          query("path")
        end

      # Z3 may return the tuples in any order so we sort them
      assert Enum.sort(solutions) == [
        {"alice", "bob"},
        {"alice", "charlie"},
        {"alice", "dan"},
        {"bob", "dan"}
      ]
    end

Finite sets implemented on top of existing theories

Small finite sets are quite useful in general, and it is useful to see how one can implement the basic axioms and operators of finite set theory in a relatively performant way in Z3. A lot of the operations are quadratic, so this is actually not super efficient, but I’m not sure I can do much better.

An example from the tests:

    require Zee3
    alias Zee3.StdLib.FiniteSet

    test "declare a set of bit vectors", %{solver: solver} do
        {:sat, model} =
        Zee3.program solver do
            s = FiniteSet.declare("s", Sort.bit_vec(8), max_size: 10)
            assert FiniteSet.contains(s, "a")
            assert FiniteSet.contains(s, "b")
            assert FiniteSet.contains(s, "c")
            assert FiniteSet.contains(s, "d")

            assert s.cardinality == 4

            check_sat_and_get_model!()
        end

        {:ok, s} = FiniteSet.extract_from_model(model, "s")

        assert MapSet.equal?(s, MapSet.new(["a", "b", "c", "d"]))
    end

Next steps

Another very useful feature of Z3 is a performant implementation of the Simplex algorithm for linear programming (optimization of a linear function under linear inequality constraints) and a not so performanc implementation of general non-linear optimizations. The current system can already handle this if one sets the right options (e.g. rounding the advanced real values, which are often presented as rationals and as roots of polynomials, which are pretty much impossible to calculate in Elixir), but I might want to add some wrappers around it.

Note that if anyone needs “normal” linear programming I already have a much better and much more performant solution in the form of the Dantzig package.

1 Like

This is not an update on the state of the Zee3 library but just a comment on the some advantages of Z3 compared to using a normal linear programming library.

For unsatisfiable sets of constraints, Z3 can give the user an unsat core, which is useful to give feedback to the users or even to relax the constraints in order to get something that’s actually satisfiable.

For those who might be interested, I have implemented a version of the MARCO algorithm to list all minimum unsat cores in a companion package: GitHub - tmbb/zee3_marco: MARCO algorithm for Zee3 · GitHub.

The algorithm is very simple, but has enough little details that it can take a bit longer to implement than one might natively expect from reading a high-level description.

Thats a security and maintenance nightmare of every person responsible for dependency management.

First of all, I think that any library must never

  1. download executable binaries
  2. access internet during compilation
  3. download not hash-verified binaries

And the way this library does it is just very wrong.

First of all, it uses httpc without any authority verification on the host. So just changing the /etc/hosts file to something like github.com my.malicious.binary.website.com would still work if the website would just return 200 with a self-signed github cert over https. If attacker has can poison the DNS for the build machine, they can run an arbitrary binary in the production.

Second, it downloads the binary for the machine it is compiled on. Again, doesn’t work when you compile on x86-64 and run on x86 for some reason.

Third, no hash verification. If z3 repository gets hacked and some other binary is returned, you may never know, because you don’t compare the hash of the file you got to the hash of the file you expected.

Fourth, compilation will fail when internet or a github will be unavailable. Imagine that your build pipeline is failing 1% of the time, because that’s how often the github fails.


The solution is to remove the vulnerable code and everything regarding this download-during-compilation approach and leave the binary download instructions in the README.

You don’t need to implement the buggy and vulnerable package manager, there are lots of tested, efficient and secure ones in the wild. Like Nix, for example.

3 Likes

I second that.

1 Like

I agree. Still, it is very convenient and it’s definitely the right tradeoff in my case. Loads of Elixir libraries have ended up picking the same tradeoff (e.g. everything that uses RustlerPrecompiled, the default packages for Phoenix build-time tooles, etc.). I don’t remember if I have documented this yet, but you can use a local executable of z3, which you can install from wherever you want. However, I haven’t implemented a conditional test on this option, so the external z3 executable is downloaded anyway… Anyway, this is a problem I can solve in a future version.

This is exactly what I don’t want from an Elixir package. One of the things taht brought me to Elixir is the extremely hands off build tool (i.e. Mix), where you add stuff to a file and things just work. For the default configuration of this library, I want things to just work and security takes a back step. There is always a tradeof between security and convenience, and I am unappologetic going all the way to convenience for this specific library. I do respect your different tradeoffs.

So, I will NOT disable the possibility of downloading the Z3 binary from github, but I would certainly welcome a PR to do the following: 1) document the config options in a centralized place, 2) add a download_z3_executable: boolean() flag or some other equivalent option that disables the binary download, 3) adding a list of whitelisted hashes/signatures/whatever for the downloads so that the source can be checked (although if you’re not compileing Z3 itself, are you sure the hashes are safe? taking the hashes from the same website that hosts the code a bit too “security theatrical” for my tastes).

And if you allow me, I will go on a small rant… I am not sure that downloading a package from Hex is safer than downloading a release file from GitHub (can’t an attacker compromise the code hosted on Hex.pm? Can’t the Hex.pm team compromise it themselves?). It’s very easy to get all worked up with security stuff and supply chain attacks, but ultimately some form of trust is required, and the hard thing is to put that trust in the right place… IMO modern software needs to be seen as fundamentally insecure, especially when written in a high-level language like Elixir: you have elixir, which is compiled to erlang, which is interpreted by a large C program (the BEAM, which famously had/has a very serious SSL bug: Erlang/OTP ssl flaws let unauthenticated attackers crash DTLS and TLS 1.3 listeners | Mallory), which runs on top of a huge multi-language program (the OS) which runs in some hardware you’ve never seen that lives on a server farm handled by people you’ve never met… And, among all of this, we’re worried about a small compile-time script (that runs on the build machine and not necessarily on the production machine) that downloads a well known binary from a rather secure source repository?

The point of this rant is not to belittle your security concerns about my package, which I do respect. It’s just a somewhat nihilistic thought about the current state of software development: we have build a tower so high that no one can see the foundations. And I’m contributing to the problem by writing yet another wrapper to yet another language for an incomprehensibly complex piece of software (Z3) which I barely understand while trusting its outputs.

It is not about convenience vs security, it is a false dilemma. It is about inconvenient and insecure vs convenient and secure. Downloading binaries from internet during compilation has these problems:

  1. Compilation now fails as often as github does (which is often)
  2. Makes it impossible to build the program without internet access.
  3. Makes it impossible to cache this binary in order to speed up build time.
  4. Doesn’t work for cross-compiled scenarios (which is an unexpected thing, because vanilla compilation generates BEAM files which are architecture and system agnostic)
  5. Has security flaws in implementation, allowing for RCE via DNS poisoning and very basic supply-chain attacks

Four out of five problems I’ve listed are problems with convenience. I think that users of your library can handle using any package manager or containerization software in order to just curl and cache the binary in their build script. It is literally one or two lines of code in any build system.

1 Like

Clearly we have very different definitions of what convenience is.

This is maybe because I am not a professional software developer and only write code to solve my own (and my business’s) problems (although I do work with some professinal devs for the more “professional” stuff), but the workflows you describe seem positively ghastly to me… As a possible consumer of this library, the idea that I would have to download any kind of software outside of what Mix already does for me, make sure it’s added to the path, etc. is a very, very big inconvenience. I envision this library as being useful for someone who wants to do some logic programming from Elixir, not necessarily someone who needs a super safe setup for production deployment (and you can get that safety once I implement the flags to avoid downloading the binary). The user I have in mind doesn’t need to know anyting about containerization.

To me, convenience is not having to leave the comfortable world of a mix.exs file and have everything work automatically and in which stuctured data is passed in a sane way between functions or sent as a message through processes without (god forbid!) ever having to use something like a unix shell where bash scripts receive and output somewhat free form plain text that is sent to some other bash scripts or weird unix utility that people insist on using as a library when the API is actually much closer to that of a command line application. To you, using the OS outside the BEAM is convenience… To me, using the OS outside the BEAM and integrate it with the BEAM is the definition of inconvenience…

I do acknowledge your arguments, and I will work to make the applications usable without downloading anything at compile time, because I’d like to have as many people using (and contributing!) to this library as possible, but keep in mind that our notions of what is convenient are very different.