/* sending: +n+1 they are ahead, ask to receive. this might happen on initial connections, if they think you might be further ahead. they have asked us to send, but they'll probably change their mind. +send +receive +n we are in sync, and neither knows who is closer to the source. since the actual receiver does not. +send +n-1 they are behind us, get the next item for them. +send !get -(n+1) they are ahead of us, stop sending. ask to receive from them (but they don't want anything from us) -send +receive -n stop sending, we know they are in sync, so no need to send notes. -send -(n-1) stop sending, but send notes sometimes. -send receiving: +n+1 they ask to receive from us, but we are not up to there. *enable send* I guess, but don't expect to send. +send +n they ask to receive from us, but we are already in sync. send a note to let the know we are in sync, if we havn't already. +(n-1) they ask to receive, but are behind us. +send -(n+1) expect them still to send to us, though. -send -(n) they just let us know we are in sync -(n-1) they are behind us, but are still getting messages faster than they would from us. get ready to send a note about where we are up to. if the absolute value is greater, and we are not already receiving, ask to receive. else if positive, turn on send (and get ready if we have something) else if negative, turn off send. --- UPDATE all negative notes are shifted -1. to say you have sequence 1 but do not want more on that feed, send -2. -1 now means "I do not have and do not want this feed" requesting 0 means "I don't have anything from this feed, please send it" and a -ve number means "I have this but stop sending, because I have a better source" you can't send -0 because if you have a better source for nothing that means you don't want it. which is what -1 means. if you have seq 1, and someone else trys to give it to you, */ OUR_SEQ, THEIR_SEQ, THEM_RECV, US_RECV onRecieveValidMessage: //after we have processed the message. if(OUR_SEQ > THEIR_SEQ && THEM_RECV) send(msg) else if(OUR_SEQ < THEIR_SEQ && US_RECV) send({id: msg.author, seq: msg.sequence}) //OPTIONAL, don't have to do this every time onReceiveMessage: if(OUR_SEQ > msg.sequence) send({id: msg.author, seq: - OUR_SEQ}) //tell them to stop sending. //else, validate the message and continue. onRecieveNote: if(note.seq < 0 && THEM_RECV) { THEM_RECV = false //they have asked us to stop sending this feed to them. } if(Math.abs(note.seq) > OUR_SEQ) { US_RECV = true send({id: note.id, seq: OUR_SEQ}) } onBeginReplication: for(var id in feeds) send({id: id, seq: feed[id].seq}) //okay I feel satisfied that is the correct logic //but how do I make this FSM pull? //I know, functional style state //For each peer, for each feed keep the state of: remote = {requested: Seq, sent: Seq, ready: msg|null, sending: bool, receiving: bool} //sending to, receiving from //also keep a map of local = {id: seq} //then I think all events can be handled sync READ: if ready!= null, the puller takes `ready` then sets it to `null` if ready was a msg, this also triggers retriving the next msg, if this feed.sent < local[id] if ready was a note, it's just sent without triggering anything. RECEIVE_MSG: //receive a message from this peer, before it's validated. remote.requested = msg.sequence //always remember they are up to this sequence. if(msg.seq <= local[msg.author]) //if we already know about this message if(remote.receiving) { remote.ready = {id, seq: - local[id]} //tell them we do not need this feed. remote.receiving = false } if(!remote.receiving) { we have asked them to stop sending, but they havn't got the note yet. anyway, remember they know this sequence. remote.requested = msg.sequence } else if(remote.sending) { this is an error, they should never send to us if we are sending. if this ever happens it's a programmer error. maybe should tell them to top sending? you might receive a } RECEIVE_NOTE: if(remote.sending) { if(note.seq < 0) //stop sending remote.sending = false, remote.ready = null, remote.requested = abs(note.seq) } else if(!remote.receiving) { if(abs(note.seq) > local[id]) //if this is the fastest peer for this feed, ask them to send. } --- TAKE 2. a lot of what we need to run a real ebt network ended up in ssb-ebt, and it should be in this repo so it's easy to extend this to other protocols. clock: {: seq,...}, following: {: boolean} connections: { : { clock: {: +/- seq,...}, msgs: [], //messages that can be sent. notes: {id: +/- seq, ...}, //notes to send. gets replaced with {} after sending. next: [id,...], //list of feeds that are in retrive mode replicating: { : { tx: bool, rx: bool, retrive: bool, //get the next item from the database sent: seq },... } } }, if this was all binary... clock: HashTable(id:seq), //10k*4 = 40k connections: { hashtable(peer_id: { //4*25 = 100 clock: HashTable(id:+/-seq), //(25~10k)*40 100~40k msgs: CircularBuffer(8k*10), //80k replicating: HashTable(id: [sent, Pointer(next), Flags(tx,rx,retrive)]), //4+4+1 next: Pointer(next), last: Pointer(last) //pointers into the hashtable //4+4 }) } 40k + N*(80k + M(10*4+9+4 = 53))