2. Notes and Stories
(You can find the original presentation written in Korean here.)
This capstone project was carried out from March 13 to June 15, 2022, as a graduation work
for CSE students.
I was inspired by LiveShare to design this service. We intended to build an web service that
has management functions and IDE features specialized in one-to-many coding classes.
Because of class rules, the presentation time is limited to 8 minutes including demo video
and the assessment criteria is unknown. Therefore, the contents of this presentation do not
show everything, but only the overall results of the project and its performance test.
For more detailed information, please visit my blog: https://blog.a2tt.me (not yet)
※ Little bit more details can be found in Speaker Notes section of this google slides.
4. Motivation - Existing Services
Already existing code sharing services
- Microsoft’s LiveShare
- Jetbrains’ Code with me
Many people can modify code
on sharer’s file system
simultaneously
7. Motivation - Required Features
- Mutually share modification of the code in real-time
- Create a QnA referencing lines of code
Real-time
8. Motivation - Required Features
- After sharing, access the codes and QnA
- Accumulate records of each student
Persistency
- Mutually share modification of the code in real-time
- Create a QnA referencing lines of code
Real-time
9. Motivation - Required Features
- After sharing, access the codes and QnA
- Accumulate records of each student
Persistency
- Mutually share modification of the code in real-time
- Create a QnA referencing lines of code
Real-time
Class-based - Systematic and continuous management system
- All participants with its own code space respectively
12. Web
React.js (from CDN → Storage)
API server
Java Spring (on AWS EC2)
Websocket server
Python FastAPI (on AWS EC2)
Runtime server
Go gin (on AWS EC2)
Runtime agent server
Python FastAPI + GCC/Python
(on AWS FARGATE container)
13. Impl. - Websocket Communication
* Share file modification
* Share cursor position
14. Impl. - Real-time File Structure
Redis
* File list and size
* Cursor position
* File
contents
22. Performance Test - Plan
https://www.speedtest.net/global-index (2022.02)
Cursor position sync latency
: 200 ~ 300 ms
LiveShare
23. Performance Test - Plan
Bitrate 60.76 Mbps
Latency 10 ms
50 tester scripts send 2 messages
per second for 5 minutes
24. Performance Test - Plan
Bitrate 60.76 Mbps
Latency 10 ms
Websocket
delay
When sending major events on a certain probability,
measure the average delay before receiving a response
Goal: avg 450 ms
Modify cursor position 40.0%
Modify file 30.0%
Save file 10.0%
Open file 10.0%
Get file list 7.0%
Create an answer 2.5%
Create a question 0.5%
50 tester scripts send 2 messages
per seconds for 5 minutes
25. Performance Test - Plan
Bitrate 60.76 Mbps
Latency 10 ms
Code modification
sharing delay
When sending file modification event,
measure the average delay
before other users receiving a response
Goal: avg 300 ms
Websocket
delay
Goal: avg 450 ms
When sending major events on a certain probability,
measure the average delay before receiving a response
Modify cursor position 40.0%
Modify file 30.0%
Save file 10.0%
Open file 10.0%
Get file list 7.0%
Create an answer 2.5%
Create a question 0.5%
50 tester scripts send 2 messages
per seconds for 5 minutes
26. Performance Test - Environment
Tester Machine
(AWS Fargate container)
vCPU 0.25
Memory 0.5 Gib
IOPS 60
Bitrate 60.76 Mbps
Latency 10 ms
Websocket Server
(AWS t3.micro)
vCPU 2.00
Memory 1.0 Gib
IOPS 100
Network Max 5 Gbps
28. Performance Test - Environment
Tester’s Private subnet
- 172.31.96.0/27
- 172.31.96.32/27
Tester → NAT → Websocket server
NAT (AWS EC2 t2.medium x2)
1. iptables mangle : packet mark by IP
2. TC : traffic shaping by each mark
3. Network Address Translation
: forwarding
29. Performance Test - Assessment
Total 119,762 messages
of transmissions and reception
- Server received : 108.6 #/s
- Server sent : 290.6 #/s
Websocket Delay Time
Average 71.80 ms
Median 22.83 ms
P95 50.40 ms
Max 3,547.97 ms
30. Performance Test - Assessment
Total 118,031 messages
of transmissions and reception
- Server received : 98.0 #/s
- Server sent : 295.4 #/s
Code Modification Sharing Delay
Average 29.25 ms
Median 22.23 ms
P95 34.28 ms
Max 785.49 ms
Hi. I’m a speaker, `a2tt`, of Together-Coding, team 2 of class 3.
Real-time code sharing service for one-to-many coding classes, carried out with student A and B.
Starting the presentation.
(Speaker Notes here!)
Have you ever heard of real-time code sharing services that are offered by VS code and Jetbrains’ products?
By using the service, the user who started it can share the files located in his/her machine and the participants including him/her can synchronize modification in the files in real-time. Thus, it provides an environment where they can develop code collaboratively.
We thought about whether these collaborative services could be used in coding classes like those of university. But, there are some problems.
Because the service provides synchronization of the files of the sharer only, when used in coding classes, all students can’t have their own code.
Also, after the termination of the sharing, although the modified files can only accessed by the sharer(Teacher, in this case), the students can’t access it anymore.
So, what should we do to make a service that can be used for coding classes motivated by the existing services?
Participants mutually share modification of the codes in real-time, and are able to create a QnA referencing lines of code.
Also, after sharing, they need to be able to access the codes and QnA used and created in the class,
and the service should accumulate records of the present condition of each student.
It should provide systematic and continuous management systems for courses and classes.
Last but not least, it should provide all participants with its own code space respectively.
Our goal is to provide a service that can help achieve better class and learning quality by complementing the existing real-time code sharing services and adding functions that can overcome the limitations of one-to-many coding classes
First of all, I’ll show you the result of our project.
(https://youtu.be/znzmxhIVmqs)
This is our architecture.
All frontend and backend servers are deployed on Amazon Web Services.
All real-time data sharing is done with websockets.
There are about 30 actions (events) on IDE.
After the websocket server processes an event, it broadcasts its response to the appropriate participants only.
All files manipulated and used in IDE are managed in Redis; an in-memory database.
We have divided the keys to make it easier to find the file list and contents, and the cursor positions of each participant on each file are also stored.
When Redis runs out of memory, there will be an eviction and the stored codes could be deleted.
In order to prevent this situation, serverless computing resource (AWS Lambda) is periodically launched and it enqueues the user IDs whose project is not accessed for a long time.
And other resources (Lambda) that consume this queue, download the project codes of each user, compress them, and backup the archive file to the cloud storage.
If a user writes a code, it should be able to execute it. In other words, there must be a terminal that can be accessed from web page.
When a user accesses the IDE, it sends a request to assign its own container.
The runtime management server (Runtime Bridge) identifies the user, and then launches a new container using prepared docker image.
The container has prepared to execute a specific language.
A python server (Agent) inside of this provides SSH relay linking user’s websocket and internal local SSH, and also processes code execution.
Because the container is not prepared for HTTPs communication, the runtime management server provides a reverse proxy to make it possible for users to use HTTPs to connect with its container.
Because of the nature of our service, we judged that the response speed of the service was important, and we conducted a performance evaluation on the response speed of the websocket server.
By applying the global fixed broadband network speed average measured by Speedtest.net, we measured the cursor position synchronization latency of LiveShare and confirmed that it would take approximately 200 to 300 ms.
So, when 50 tester scripts send 2 messages per seconds for 5 minutes with the same network speed, we measured the delay time from the departure of the each message until users received its corresponding response.
Our first performance goal is related to measuring the overall delay time of the service when sending major events on a certain probability.
The second is related to measuring delay time of sharing file modification data, which is one of the primary features of our service.
The tester machine was set to the minimum operational specification.
In order to configure an independent test environment, we launched docker container (AWS Fargate) in two private subnets.
In order to apply traffic shaping, we made the traffic from testers go through NAT server.
In the NAT server, by using `iptables mangle` command, all packets from the private subnets are marked by each private IP address, and for traffic going outside from NAT, bitrate limit and delay are set for each mark.
The test results are as follows:
When primary events were sent 2 times per second for 5 minutes, the average delay time of 71.80 ms was measured and the target value of 450 ms was met.
In addition, code modification sharing has an average delay of 29.25 ms, achieving results that met the target average of 300 ms.
This is the end of our project. I’ll wrap up the presentation. Thank you.
