% vim: set tw=78 sts=2 sw=2 ts=8 aw et ai:

The communication between edges and supernodes on the discovery stage was designed considering two options. First option was to integrate the SNM messages semantics in the existing N2N protocol. This solution would have involved that a supernode would have to handle SNM requests and responses under the N2N protocol as well. An edge-node would only connect to advertise addresses of supernodes, meaning that a coordinator assisting the edge bootstrap must keep track of all other supernodes advertise addresses as well.

The second option, that was chosen for the current implementation, implied using the SNM protocol for edge-supernode communication as well. The difference from the first option is that an edge must find the advertise addresses of supernodes coordinating its community. It has the advantage that it doesn't alter the existing N2N protocol and it keeps simplicity in the implementation. Sending back-up supernodes addresses to an edge was implemented by filling the supernode registration acknowledgments.

An edge-node starting for the first time, will begin in a discovery state, meaning that its main target will be to find the supernodes that will coordinate its community. There are 2 possible situations:
\begin{itemize}
\item the edge-node is initiating a new community and it will have to choose the supernodes that will coordinate it
\item the community already exists and the edge-node will discover the supernodes that are coordinating it
\end{itemize}
If the edge-node is initiating a new community, it will request the SNM address and communities number of every running supernode. After all information is collected, it will choose the supernodes that will be coordinating the new community. A supernode will be ranked based on  its communities number and its response time. Next, edge will start an advertise requesting stage receiving the advertise addresses of supernodes it has chosen. After all addresses are received, it will become ready from communication with community peers.

If the community already exists and the edge-node is starting for the first time, it will start discovering its community supernodes. The supernode discovery process will begin with sending requests to supernodes for that the SNM addresses were provided as command line parameters  (e.g. \emph{./edge -l  a.b.c.d:5645}). If a receiving supernode is coordinating the community, it will advertise its N2N address along with those of its counter-parties supernodes and the edge will become ready for processing messages from its peers. Otherwise, the supernode will send its communities number and the SNM addresses of all known supernodes.

After getting the complete list of coordinators' advertise addresses, the edge-node will be ready to process any incoming messages from its peers. Registration messages will be send to each supernode of the community in a round-robin fashion, that will help supernodes maintain the complete lists of community peers. A supernode registration acknowledgement will contain the advertising addresses of other coordinating supernodes as well. If a supernode will voluntarily decide to coordinate the community, its address will be advertised in the register acknowledgments. The list of community supernodes is saved in a file and loaded every subsequent next start-up.

The rank of a supernode is computed based on the response time and the load level that is represented by  number of coordinated communities. The formula used in ranking is:
\[
Rank = \frac{ResponseTime \cdot 100}{n},
\]
where \emph{n = MAX_COMM_per_SN – CommNum} if \emph{CommNum $<$ MAX_COMM_per_SN} or \emph{n = 1} otherwise.

The formula was chosen in order to offer a greater chance to supernodes having a lower number of communities. First \emph{MIN_SN_per_COMM} supernodes are chosen in from ascending ordered set of rankings. All supernodes advertise addresses will be kept in a file, each entry having the form \emph{IP Address:Port}.

All SNM messages added apply for edge-supernode communication. A SNM request initiated by an edge will have the \emph{E} (edge) flag set and will contain its community name. During supernode discovery stage, the edge-node will start sending requests for SNM addresses until the community supernodes set will be established. If the community is new, the edge will send requests for advertise to all the supernodes it decided to use.

A SNM response payload received by an edge-node may contain either SNM addresses or advertise addresses. The difference is given by the \emph{A} (advertise) flag, in which case the edge will use the addresses to register to supernodes. Upon receiving  such a response, the edge-node will stop quering for other supernodes SNM addresses and it will become ready for communicating with peers.

A SNM advertise message can only be received by an edge-node during discovery stage and only if it had previously requested it from a certain supernode.