Three midnights on New Year’s Eve at LINE

For developers at LINE, there are three major New Year’s Eves each year: First the New Year comes to Japan, then an hour later it’s midnight in Taiwan, and an hour after that the people of Thailand ring in the New Year. For each one, LINE experiences a major surge in traffic, which forces our engineers to be alert and ready to act. 

To learn first-hand what it’s like to go through those three big surges, we talked to Hyunsu Kim from the LINE PLUS Service Engineering Team, who was the project manager for the 2021 New Year’s Traffic Response Task Force.

New Year’s traffic response

Q. Please share what exactly entails LINE’s annual New Year’s traffic response.

A lot of people send New Year’s greetings to their family, friends and work colleagues right around midnight on New Year’s Eve and early on New Year’s Day, and LINE is one of the most popular ways of doing so. In fact, New Year’s is our biggest traffic surge of the year, in terms of greetings and messages. In fact, in the early days of the LINE messenger, sometimes that surge caused service failures because we could not handle this large volume of traffic. In order to prevent such failures, we inspect the system and fix potential issues starting in September each year. If it is deemed necessary, we expand our infrastructure or adopt other solutions to improve the situation.

Q. How is the New Year’s traffic response task force staffed?

Since a lot of people participate in the New Year’s traffic response, we create a task force rather than a team. This year, around 200 developers based in Korea and Japan worked hard to ensure the smooth handling of traffic on New Year’s. First, we select a leader to take charge of the New Year’s traffic response for the year. Then, that person organizes the service components that are required for the New Year’s traffic response and starts preparing for New Year’s Eve together with the teams and service owners who are in charge of each service component.

 

Q. What are some of the tasks and important factors that the New Year’s traffic response requires?

The details of what needs to be done varies a lot, depending on a service component’s aspects and features. But, in general terms, each service component needs to estimate the traffic volume it expects to handle that evening and checks whether we have sufficient infrastructure to handle it, such as servers and network.

On New Year’s Eve, we come to the office at around 4 p.m. and work until 3 a.m. Because the traffic is usually high between 9 p.m. and 11 p.m., we restart the entire Talk-Server, which is the core service of LINE messenger, at around 5 p.m. in order to avoid potential issues. In addition, we continuously check whether everything is being performed correctly, such as adjusting the log level and turning off the push tracking function (which does not have a direct impact on the service operation, but could slightly improve the capacity of the service).

The most important factor in the New Year’s traffic response is inspecting whether the LINE Messaging service is handling the traffic without any issues. We pay extra attention to the benchmark test (BMT) process to check whether the anticipated messaging traffic is being handled sufficiently and we make corresponding improvements if necessary.

Q. Cooperation with LINE’s other offices must also be crucial. How do you all work together?

I focus on keeping communications smooth when working with the other offices. We hold a number of conference calls during the preparation process, and use interpretation services to make sure everything is clear. In addition to the conference calls, we open a separate communication channel where everyone can talk and share. This channel is in English so that everyone can understand what is being said.

Q. The work environment has changed a lot due to COVID-19. What changed about the 2021 New Year's traffic response?

Because of COVID-19, most of the 2021 New Year’s traffic response was done from people’s homes, not in the office. In the past, when we were all in the office together, we could do most things verbally because the work mostly involves checking various things over a short period of time. 

Working from home, however, required us to arrange the means of communication in advance. We created a conference room for each component and a separate conference room for emergency discussions in the event of any failure. We pre-arranged contact methods to ensure a smooth communication environment.

As we prepared a lot in advance, this year’s New Year’s traffic response went well, without any issues. Afterward, we had some assessment meetings and made a list of improvements that could be made in the work-from-home environment. Personally, I was pleased to know that many of my colleagues who have been working on the New Year’s traffic response for several years were able to spend this New Year’s Eve with their family, thanks to working from home.

Q. Anything else you’d like to share?

We were able to finish our New Year’s traffic response smoothly and welcome 2021 thanks to a lot of people who worked hard and did their jobs well. I’d like to thank them and express my sincere gratitude for every LINER who worked hard. I wish you all happiness and good fortune in 2021! (And I hope we will soon be able to live without wearing face masks!)

 

 

• Read the LINE Engineering Blog post related to New Year’s traffic response [Click here]
• Watch the LINE Development Archive video related to New Year’s traffic response [Click here]

 

 

 

※ Please note that the details described in this article may not apply to all offices.