Page MenuHomePhorge

Internal design considerations - users
Closed, ResolvedPublic

Description

So, we have users table with username and id columns. I think we should keep it as a cache for user information e.g. real name, maybe photo, and maybe some preferences in future.

When we start creating more database tables (channels, invitations, etc.) we'd need to decide if we should internally use user id or username (in case of Kolab it would be an email address, but maybe not always). I suppose we have to use username as a user identifier:

  1. to really make the users table optional
  2. to make user identifiers unique and constant across servers (federation, etc.)

Details

Ticket Type
Task

Related Objects

Event Timeline

I thin it would be very nice and interesting to see if we can run the chat functionality without a database connection at all. The authentication mechanism could be responsible for retrieving the username to show for a given login (defaulting to the login itself) along with "nice things" like the user's image to show, etc. This way, when we go to integrate with the larger Kolab stack, that user information does not need to be synchronized also to the chat database. Wherever they auth against would also give the chat service the user's metadata like username, picture, organization, location info, etc. This has obvious overlap with the user's addressbooks and identities; amazing would be allow the user to select from their identities as configured in Roundcube ... we can get their later, but should keep that kind of integration in mind now. Mostly so we don't end up duplicating that in the chat service directly.

Where I do see some sort of storage being desirable / required is for off-line / delayed message delivery (user A sends user B a message, but user B is actually offline at the moment; when they log in, they should receive those messages) and for chat logs (so you can see the recent chat history when joining a channel, for instance), and channel connection persistence between logins. But none of that really screams out "relational database" to me. Recent chat history could be maintained live in the channel itself, sent to the client on connect (when the channel goes away, so does that recent history). Storing logs for long could be done via an actual logging mechanism (tbd). Off-line delivery and maintaining channel state could be SQL, but is probably even better done using a key/value store, and Elixir ships with one of those built-in (ets / mnesia). The channel state could be managed in the theoretical clientsync:%username channel, and when the last of the user's connections expires, their channel state could be saved out to that same key/value store for later retrieval when the clientsync:%username channel is created again.

Soooo .. tl;dr -> let's see if we can't do this without any SQL server at all. I bet we can!

@seigo, I'm not sure we can do it without SQL, how about invitations? I'm thinking of a case of "I'm logging in, give me all pending invitations". I don't think we can store invitations inside a chat. I'm used to use relational databases and can't imagine how to use key/value store here.

Key/value store should be enough for that. Mnesia (which comes with Elixir/Erlang) uses a tuple as the definition for a table, with an implicit primary key, you can index and search on any field in the tuple. So it is very much like an SQL database in that it has tables, primary keys and optional secondary keys. But it is built into the language and runtime itself, supports RAM-only and on-disk storage as well as clustering if desired. It's a very nice tool to use for "lightweight" database needs (e.g. I wouldn't necessarily try and map a dozen tables with complex relations / refint requirements, or a dataset that was expected to grow to dozens or more GB and require long term storage), as it is easy to use, fast, well integrated and means one less dependency.

So in this case, I would probably have the tuple include the invitor and invitee's user ids as indexed members, and also the channel the invite applies to, an optional small message from the invitor, ttl, etc. For the primary key ... this could be just a generated UUID or similar, such as by using Ecto::Uuid.

You can see nice docu on using mnesia here: https://elixirschool.com/lessons/specifics/mnesia/

.. oh, and if we really need/want to, we can always migrate to a full SQL RDBMS in future. The storage/lookup code is not really so big. The reason for choosing mnesia in this use case if because it is largely ephemeral data, speed is nice, and fewer dependencies / setup routines is never a bad thing.

remembered Phoenix.Token just now -> https://hexdocs.pm/phoenix/Phoenix.Token.html

This is absolutely perfect for our needs here. There is even a rather nice little auth example with Phoenix.Socket for use with channels in that doc above! Has ttls, signing, the whole kit!

We already use Phoenix.Token for user authentication tokens. But I don't understand for what you propose to use it. User metadata? I suppose not for anything else?

Sorry, I skipped some details. I was thinking about invitations in specific, and how we could use a Phoenix.Token as part of the invitation metadata to make authorizing the invitation when it is used easy/simpler. I may have actually put this in the wrong ticket even. Indeed, this belonged on T2118 *sigh*.

Amnesia is quite nice, and makes using mnesia rather more bearable from Elixir. Let's try it!

machniak claimed this task.
machniak moved this task from Ready to Done on the Kolab Chat board.