I’m stuck with a problem and would really appreciate any feedback or ideas you might have.
The problem
We have a Phoenix app deployed on Heroku and it’s working well. We have a requirement to integrate with a third-party service over HTTPS. That service has provided us with a client certificate that we should use when calling their end-points. We are using HTTPosion to do this and locally in development it all works fine when we read the certs from a file.
However, we need to get these certs deployed to Heroku. We don’t want to add these files to the repository as they are sensitive and Heroku doesn’t allow file storage separately for you app to read.
Possible Solution
One option we are trying out is to put the content of these certificate files in environment variables in Heroku (like all other config) and then reading those strings and using them instead of files. However, we have been unable to get this to work. Here is what the code looks like that works with files:
When we try to use the certs from strings we get errors, here is the http_options function we tried with certs inline (snipped) for testing:
def http_options do
public = """
-----BEGIN CERTIFICATE-----
MIIC0DCCAjmgAwIBAgICAyIwDQYJKoZIhvcNAQEFBQAweDELMAkGA1UEBhMCR0Ix
...
L7FNVw==
-----END CERTIFICATE-----
"""
private = """
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,3C6A3A4613A1C924
OSJjqiIcqVIvljlzOmYGq3MARW86TPUouLymH2aZ4yIxJ2coQAZBm7cy3gStUJrh
....
vGac+7HotjLzeY5rmuijFOdbg+EUCfJo/D6DmlNs/2a2deu2RDohYQ==
-----END RSA PRIVATE KEY-----
"""
[
recv_timeout: 30_000,
ssl: [
cert: cert(public),
key: cert(private, "a-password")
]
]
end
defp cert(text) do
[pem] = text |> :public_key.pem_decode
:public_key.pem_entry_decode(pem)
end
defp cert(text, password) do
[pem] = text |> :public_key.pem_decode
:public_key.pem_entry_decode(pem, String.to_charlist(password))
end
With this, we get the following error:
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: {:options, {:cert, {:Certificate, {:TBSCertificate, :v3, 802, {:AlgorithmIdentifier ......
It looks like HTTPoison is just handing off the ssl args to the erlang ssl module. Looking at the erlang ssl module, the typespec for ssl_options() has the following types for cert and key:
So it looks like your key needs to be a two item tuple that contains an atom with the key type and a binary for the key itself. It looks like AlogorithmIdentifier in the error is learning towards this as well.
Thanks @kylethebaker, that makes sense now that you say it. My erlang isn’t great and I find the docs a little hard to understand. I’ve made the following change as you suggest but still getting the same error, is this what you mean?:
I’m not sure I’m doing the right thing in the cert functions for decoding the certificates, I saw this online somewhere (can’t remember where) as a step that was required.
Both the key and cert options are expecting a public_key:der_encoded type. There is public_key:der_encode() that seems to be what you need.
The parameters it accepts are kind of tricky to figure out. The first parameter is Asn1Type, which appears to the key type atom again (:RSAPrivateKey or :RSAPublicKey), and the Entity param is just a generic term that is an “Erlang representation of Asn1Type”, which indicates to me that it varies based on whatever Ans1Type you are using.
Looking at the type specs, there are rsa_private_key and rsa_public_key types, which are both records (#'RSAPrivateKey'{} and #'RSAPublicKey'{}). Records are the erlang equivalent of module structs. I can’t find where these records are defined in the code, but I did find this: Examples — OTP 29.0.2 (public_key 1.21.2)
It shows pem_entry_decode as returning an #RSAPrivateKey record. So I think something to try is making your opts be:
** (MatchError) no match of right hand side value: {:error, {:asn1, {{:badmatch, {:Certificate, {:TBSCertificate, :v3, 802, {:AlgorithmIdentifier, ...
(public_key) public_key.erl:266: :public_key.der_encode/2
(platform) lib/.../web_services.ex:124: MyApp.WebServices.http_options/0
Line 124 if the stack track is the cert config line:
Sounds like the certificate is in the wrong format? I remember erlang being picky about it, if you can create a new one and test and show it then maybe we can see something about it?
I think I may have figured it out, but you will need to test with your info. The cert ans1 type should be :CertificateRequest instead of :RSAPublicKey. I was able to get a successful request with the test code below. If I tweak the cert/key options I can get similiar error messages to what you were getting before. I don’t think the ssl cert is doing much in this example request, but it doesn’t throw any errors at least.
I generated the RSA keypair with ssh-keygen -t rsa -C "your_email@example.com" and then created the certificate with openssl req -new -key priv_key -out pub_cert.csr
EDIT I just realized that I was using a :CertificateRequest instead of an actual :Certificate before. I generated an actual cert using openssl x509 -req -days 365 -in erlang_cert.csr -signkey erlang_id_rsa -out erlang_cert.crt and it still seems to be working. I am using whatever ANS1 types come from the decoding rather than hardcoding them, so the code doesn’t change, just the certificate.
defmodule ErlangSslTest do
def priv_key, do: """
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,E9BF6F551AB6C6C8E2ACB1FD9B7BF860
JkeMooohdh8OT0y2DuRIZqcotwUQmwkAYVy6LKSRQ/bpfZL1p1jQc60S7BE4e0AT
QtjmxPi26VQjYiR13/v/PUfoP5bUwz3aEWbBARBCE++Fl92dB8jH1e+DOR3b9MZ/
cWXI6Qi/TKCMnl9JG+mh4l+THBspTLKC3qj1RM0Fdg4Zh1NtxKmmY/zXgx+ss7oZ
r8sAjLnuqT9mttjp+eC8ycgWIXySRsiuIE/r89ySny8rvLcOQXOc3f3SDLtpZhgM
p1VYGYZaa0oKvuPHUu1rGKxd0rLrW0tFZ1S2G9DQGRTAD1NB8CrHlNQXChKHBbOB
4nd2Bd2Awi4Ut9otrX8KMRaPe6lwJivOu+j8GaVyukZ+FGfHD5riWnWdaUny3Gfu
B+plHs0DmXYfl35+Cm0ZvyorQXXrPGodhqAeWXEfWeaLqBqv8WafGcKaYng7TEbh
MXzzZ5TxZG5jAzAutuCDXzV5ZXXe1CYgJEPKdlb7favK7UU9OziqX/Xj6fye8Sf/
ybiUYdXJFWZVPvVqeFxwwURLIhf21Lg9J7LBt3wek9GrezTGou3JXMzr5/66ENTB
UEPSWtZtKsLsfKWLrsf1I1WHSVbOMUhln/7t+7wOM4bb+Km3ZfqjE6RBxNCdcRQW
u1Fxg1Bxs8O6nNpXT+YPAg7yA+gmYXdkoAinrwswKtQu+cOQhl0ZwaW9u7OTTsAC
ZVWlp7KgpVwJ6yM5HfttdCt/RcWkU95NfhgGt3SVRNUkpk9gUoMsVyav5783Tg6t
nfigUKKlOrDqMOUQJDP5bTDkWdWTLcmH+4AGEgGJVKuE0kF3uiU0qgU4IUH+L0oi
1QQpdEWR+2zYdGmDJBWEyevWvoqxcCq+YTVbGxlqFjO5KImR5nmEDYz7M1/FyqXX
Dp56X75HpoeL+zuDBl+evwKh5JiBDcvOSL7+st77gTaW1kAIrC31ItVOkuWKU259
cXwq8SQ1Xdg8ihcxfUQmvS1BksRgVoh2G1YZK6mrIeKMG+yPnncxV3rBe65K24/U
6oU9GNGenQx9dzvp2nhPtMzgDOnwhdvMOXnQ5iMyjWPRtEtLEFzz+dEluZwdYibv
ZZnJvCGO3ug83guFvuQshuHy2DyipdyazKhmNR21yxUXTuV9ezzGQIjgf0viONEQ
pIb65h16ynx4yWVNyfdTgiKy1TK8ReYzIYDHuP3k6yt/soGHNEwb32yRbzIeq1wq
S8iSJNuij809rxgDJlvXiVPTu6a93itbUoD1W+kC+dI5wvA9uAKTSOZDqII5K2bw
BidYt2NxIfnmzBhAR7y75TAL93UNInEw0efAenQ7I2UqMZOW2NXCrxoToYkbWXQz
AX/VvejcghINZAArPyYmSrjqi+6YhJu9BJs1X77rJbmveVvL2aH/r9XsXGxEqVGB
XgpYR7xwN9AmTkt8cetUUHcJ1XBU1OC9Pc4FUS0qIQ1UVjxWWP5yUiYmx/z6g5q3
jPC8lIPjtdZYxC5tewMM5arl3eh1H9URgq5Eczg5FxpybmAYyVrtOBiVXOdD40Bf
DUotXu7FD/w3UEeq35dVjvOIlUq38z3Ll4PjeUrcTLAsLcUfViTHrh0Aq+Cyxg13
-----END RSA PRIVATE KEY-----
""" |> String.trim
def pub_cert, do: """
-----BEGIN CERTIFICATE-----
MIIDBjCCAe4CCQDDhq+iQYSezjANBgkqhkiG9w0BAQsFADBFMQswCQYDVQQGEwJB
VTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50ZXJuZXQgV2lkZ2l0
cyBQdHkgTHRkMB4XDTE3MDkxNDIxMjM0OVoXDTE4MDkxNDIxMjM0OVowRTELMAkG
A1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoMGEludGVybmV0
IFdpZGdpdHMgUHR5IEx0ZDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AL2y30DlnBpy0XACIqgdj1ZroIf1kl5tIuM4nOxL+piigvU/0nSCZl1tj+kvtHyC
rOkwg7SoPn9xoDbikvDX8zr2OwOGI01zXn8zUp1jukLzPr0hcgkjxb+fTgSmTNxI
fiZ+WYUnCS1TcKmgL50uKACXSTWt9ZZaDlZZ0Ta8gPh7LFyD2ie5rxyQyESTNykv
LirUx02nuCuF2VaF7lPGz+cSxZHKy+OgNvHtHWDUCD3e5wIbYStVdnpetFo5VsiA
v+/r5fbBCjanYoJTXecg5swVhxeuLi3LpjE4syOTJbpu2xtTA0KjMZhXcaMbZ6kn
SkpaFIgHFHq/WYwDYDj14B0CAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAN8IWdVua
/8ag/7EUxwTWAJn4qYQHXo9tmWJxuYy1Lf5PVcNTqVcEOQG2ZJBi7F6ADHgbLc2N
e5fsLca2VYpQ6LvEi9sHun06t7KI6PPB3/EzesTMJzAr5aWK9NdcqjTEtjshZNnt
TIZwFrzX7XSZzpBijS0tEItYbJuraYC2LWuFhJtzTiStK4NrzfyFJVG0oQ1V3SU4
n9EAFBu23FS52qfxrTauTCNYL43V+ha2gvm70VJAQIVpQs4Z3MgSyKGYr+LAIuKB
dtY/lEgcUYhXY5Dzqt7DotJLRM1i/dWUrxLjLw0oyMi4EyQn2j4GmN4nlr/yBPdM
EVCBwVsZ06HKtA==
-----END CERTIFICATE-----
""" |> String.trim
def decode_pem_bin(pem_bin) do
pem_bin |> :public_key.pem_decode() |> hd()
end
def decode_pem_entry(pem_entry) do
:public_key.pem_entry_decode(pem_entry)
end
def decode_pem_entry(pem_entry, password) do
password = String.to_charlist(password)
:public_key.pem_entry_decode(pem_entry, password)
end
def encode_der(ans1_type, ans1_entity) do
:public_key.der_encode(ans1_type, ans1_entity)
end
def split_type_and_entry(ans1_entry) do
ans1_type = elem(ans1_entry, 0)
{ans1_type, ans1_entry}
end
def get_cert_der do
{cert_type, cert_entry} =
pub_cert()
|> decode_pem_bin()
|> decode_pem_entry()
|> split_type_and_entry()
{cert_type, encode_der(cert_type, cert_entry)}
end
def get_key_der do
{key_type, key_entry} =
priv_key()
|> decode_pem_bin()
|> decode_pem_entry("password")
|> split_type_and_entry()
{key_type, encode_der(key_type, key_entry)}
end
def poison_options do
{_ans1_type, cert} = get_cert_der()
key = get_key_der()
[
recv_timeout: 30_000,
ssl: [
cert: cert,
key: key,
]
]
end
def test_poison do
{:ok, response} = HTTPoison.get(
"https://httpbin.org/get?foo=bar",
%{},
poison_options()
)
end
end
I wonder if an Elixir wrapper around the ssl/public_key erlang libraries would be useful. There is definitely a pain-point here, but I’m not sure if its from the nature of the domain (PEM, DER, ANS1, x.509, certs, keys, ohmy), in which case a wrapper won’t be too useful, or if it’s because the erlang libraries are mainly plumbing with little porcelain. My concern is that the elixir wrapper would essentially be a one-to-one mapping with small helpers and very slight differences, in which case the benefit is small compared to the maintenance burden and potential bugs (which can be especially bad because of the security implications).
I suppose the typical use case is just providing the key/cert filenames, in which case it seems like things “just work”. The files are typically PEM formatted, so it makes sense that there can be single interface that hides all of the implementation, whereas if you are providing a cert from memory then it’s harder to make assumptions about whether or not it is PEM or DER. Perhaps rather than a library what is really needed documentation covering this use case, which hopefully this post can provided in a sense.
def init(pem) do
{:ok, %{
public: ECC.Crypto.parse_public_key(pem),
private: ECC.Crypto.parse_private_key(pem)
}}
end
def start(pem, register_name \ nil) do
if register_name do
GenServer.start(MODULE, pem, name: register_name)
else
GenServer.start(MODULE, pem)
end
end
def start_link(pem, register_name \ nil) do
if register_name do
GenServer.start(MODULE, pem, name: register_name)
else
GenServer.start(MODULE, pem)
end
end
def handle_call(:get_state, _from, state) do {:reply, {:ok, state}, state}
end
def handle_call(:get_public_key, _from, state) do if state.public do
{:reply, {:ok, state.public}, state} else
{:reply, {:error, :no_public_key}, state} end
end
def handle_call({:sign, msg, hash_type}, _from, keys) do {:reply, {:ok, ECC.Crypto.sign(msg, hash_type, keys.private)}, keys}
end
def handle_call({:verify_signature, msg, signature, public_key, hash_type}, _from, keys) do result = ECC.Crypto.verify_signature(msg, signature, hash_type, public_key)
{:reply, {:ok, result}, keys} end
end
defmodule ECC.Crypto do def encode_der(ans1_type, ans1_entity) do
:public_key.der_encode(ans1_type, ans1_entity)
end
def parse_public_key(pem) do
# Expected result:
# {#'ECPoint'{}, #'ECParameters'{} | {namedCurve, oid()}}
try do
pem_keys = :public_key.pem_decode(pem)
# pem_keys |> IO.inspect(label: "parse_public_key: ")
# 'ECParameters'{
# version, % integer()
# fieldID, % #'FieldID'{}
# curve, % #'Curve'{}
# base, % binary()
# order, % integer()
# cofactor % integer()
# }
ec_params =
Enum.find(pem_keys, fn(k) -> elem(k, 0) == :EcpkParameters end)
# |> IO.inspect(label: "EcpkParameters")
|> put_elem(0, :EcpkParameters)
|> :public_key.pem_entry_decode
|> IO.inspect(label: "ec_params")
pem_public =
Enum.find(pem_keys, fn(k) -> elem(k, 0) == :SubjectPublicKeyInfo end)
# |> IO.inspect(label: "pem_public: ", limit: :infinity)
|> elem(1)
# |> IO.inspect(label: "pem_public: ", limit: :infinity)
ec_point = :public_key.der_decode(:SubjectPublicKeyInfo, pem_public)
|> IO.inspect(label: "ec_point A", limit: :infinity)
|> elem(1)
|> IO.inspect(label: "ec_point B", limit: :infinity)
|> elem(2)
|> IO.inspect(label: "ec_point C", limit: :infinity)
# {cert_type, encode_der(cert_type, cert_entry)}
# {{:ECPoint, ec_point}, {:namedCurve, {1, 3, 132, 0, 35}}}
{{:ECPoint, ec_point}, ec_params}
rescue
_ -> nil
end
end
def parse_private_key(pem) do
try do
# Decodes PEM binary data and returns entries as ASN.1 DER encoded entities
:public_key.pem_decode(pem)
|> Enum.find(fn(k) -> elem(k,0) == :ECPrivateKey end)
|> :public_key.pem_entry_decode
rescue
_ -> nil
end
end
def sign(msg, hash_type, private_key) do
try do
# sign(Msg, DigestType, Key) -> binary()
:public_key.sign(msg, hash_type, private_key)
rescue
_ -> nil
end
end
def verify_signature(msg, signature, hash_type, public_key) do
try do
# verify(Msg, DigestType, Signature, Key) -> boolean()
# Msg = binary() | {digest,binary()}
# DigestType = rsa_digest_type() | dss_digest_type() | ecdsa_digest_type()
# Signature = binary()
# Key = rsa_public_key() | dsa_public_key() | ec_public_key()
# public_key |> IO.inspect(label: "\n -> verify_signature with: ")
:public_key.verify(msg, hash_type, signature, public_key)
rescue
_ -> nil
end
end
end
=== lib/demo.ex ===
defmodule Demo do @msg “Elixir”
We want to match the definition of ssl_option() here: ssl — OTP 29.0.2 (ssl 11.7.2)
Which we do with the above functions. Let me know if this simplifies your code.
Thanks for the post and it totally worked for me. As I need to get the key into an ENV variable so I can use it with Heroku instead of reading it from file.
One cavet to note is that pay attention to the output of find_key method. hd() only returns the head of the list. In my case the pem file had :EcpkParameters for the first few lines and I actually needed to grab the tail of the output that specifies :ECPrivateKey in the tuple. Then it works like a charm as value to the key
