Hi, i plan to build 2 phoenix applications, let’s say appA and appB. AppA will be having multiple phoenix containers (to match scalability requirements) in a cluster. AppB (single node) will maintain state in ETS tables. ETS tables on AppB can have at max 5gb of data. I am looking out for a way to…

[image] chaitanya: i can try with HTTPoison. but as that is a http protocol, when i try with a million concurrent connections, it can be slow. How much data do you need to be sent? Why would there be millions of concurrent connections?

Can you connect A and B nodes in the same Elixir cluster via Distributed Erlang? In this case A node could just send messages to B node and vice-versa.

the data to be fetched will be around 20kb. when i say the statement “when i try with a million concurrent connections”, i mean the app will be access by users from web and the concurrency is expected to be heavy.

@alvises initially started with the intent of storing the data in ETS tables in a single application cluster. I am not sure how to handle the network split scenarios and hence thought of making it into 2 applications. one as stateless and another with state. hence trying the distributed erlang mig…

Tell me If I get it right: You want A to be replicated without any state (not even cache to avoid issues during a network split), and the only source of truth is B, a single node. If B doesn’t make any particular transformations and processing, instead of making your own db in AppB, I would simply …

@alvises i will be storing my data in postgres and it will always be my source of truth. the data which i am planning to store on AppB is for faster access (instead of making numerous round trips to database). However it seems in this case for me to validate the cached data (to find if that is stal…

[image] chaitanya: the data which i am planning to store on AppB is for faster access If you need faster I/O than postgres, just use redis (with redix library) instead of re-implementing it yourself on AppB.

IMO your easiest option would be for every app server in the cluster to maintain their own local cache through ETS. All local caches will warm up eventually. Otherwise you’ll have to deal with a lot of complexity. Not worth it going for Redis IMO. If your DB server is good (no need for highest-end)…

As a more general answer, in decreasing order of efficiency: Don’t split your app into multiple apps. (after all, the fastest kind of communication is no communication.) Run your applications in the same cluster, and use distributed erlang to communicate. (You can now use Erlang’s battle-tested di…

thank you @Qqwy , @dimitarvp and @alvises for your inputs. As stated earlier, following are the conditions i have in designing the architecture for this application. users when login and visit a page, data need to be fetched from database and served. if around 50,000 users login at the same time, t…

Efficient way to establishing communication between Phoenix applictions

Questions & Help Questions

LostKobrakai January 21, 2020, 5:12pm 2

How much data do you need to be sent? Why would there be millions of concurrent connections?