Meet the Global LINERs
Engineering
Tanino Mitsuhiro / Cloud/Infra / LY Corporation
Verda is a private cloud platform that LINE developed in-house by our talented team of international developers. One key member of that team is Tanino Mitsuhiro, who agreed to talk to us about why he decided to work for LINE and the Verda team as a software engineer, as well as about LINE’s general working environment.
A quick three-line summary
- Tanino considers LINE’s work environment to be ideal for a software engineer.
- Working on Verda, LINE’s in-house cloud platform, is a unique engineering experience.
- If you’re looking to enjoy your career as a software engineer, he highly recommends joining us at LINE.
About LINE
Q What did you do before you joined LINE?
Before LINE, I worked for a Japanese electronics manufacturer, where I was responsible for Linux-related tasks such as evaluating the performance of Linux as an operating system, providing support services to users, and developing new functions that the company could provide to customers. I was also involved in the OSS community with activities involving Linux, such as developing Linux kernel functions, OpenStack storage management functions, and Kubernetes storage management functions.
Q What made you decide to join LINE?
First of all, I wanted to engage in OSS-related development. Secondly, I looked for a work environment and positions that would allow me to grow as a software engineer over the next few years. LINE checked all my boxes. LINE focuses on the development, improvement, and operation of OpenStack functions, which I used to be engaged in, and offers these functions to many users through its own large-scale Kubernetes-based platform. Moreover, LINE offers services to users around the world and has a global team structure, which attracted me. Again, the most important factors I considered when deciding to join LINE were about working as a software engineer, continuing to grow, and contributing to the company. LINE’s employees can choose different career paths other than management and are evaluated accordingly, which was a critical factor for me. At present, I am handling a big project with some members from my team while doing technical work as Tech Lead in my field.
Q Can you talk about the work environment at LINE?
In LINE, especially on my team, there are people from many different regions, including Japan, Korea, India, and Taiwan, all working together. We have a very modern work culture and English is used as the common language in the workplace. This type of work environment is unusual in Japan, and I like it so much. I have been involved in open-source communities too, and I feel that LINE’s workplace atmosphere and communication methods are similar to those of open-source communities. At LINE, I can work at my own discretion and control my work hours independently. Currently I have to work from home because of the COVID-19 pandemic but still I can work as smoothly as if I were at the office. I think LINE’s work environment is influenced by the culture of freedom that encourages employees to work efficiently.
About work
Q Can you talk about your team?
My team is in the division called the “IT Service Center,” which provides comprehensive services for LINE’s infrastructure. The IT Service Center is responsible for the development, management, and operation of physical and cloud servers, networks, storage, and the groups of software that control them, all of which are essential to provide various services including LINE apps. The Verda department, which belongs to the IT Service Center, provides Verda private cloud platforms. Besides virtual servers offered through OpenStack, the main software platform, Verda provides private cloud platforms and manages services such as software load balancers, storage platforms, and container-based PaaS and DB. All of these things are collectively handled as Verda platforms. My team is dedicated to developing platforms using OpenStack at the Verda department, and I am responsible for managing systems for the physical servers provided at LINE.
Q What is your daily work like?
There are two different functions for managing a Verda private cloud—one is for providing virtual machines and the other one is for providing physical servers. We manage virtual servers using the OpenStack function while developing in-house software to manage the physical servers provided to software engineers in the company. I am in charge of maintaining the management function of the physical servers and adding new functions to them in response to requests from employees who use them. It is a great opportunity for me to be able to develop and operate private clouds, and I am happy with LINE’s work environment, which allows me to take a chance on something new by utilizing my experience and knowledge as an engineer. I have many colleagues who are experts in various fields, which makes me feel that my workplace has a lot of opportunities to further improve my skills.
Q What are some important things you need to perform your duties?
My team always has various tasks that must be tackled simultaneously. I tend to just concentrate on my own work, but I believe that it is important to work in the order of priority with a broad view, focusing on the common goal as a team. I also spontaneously provide as much support as possible to my team by conducting document and code reviews. Another important thing about my work style is to ensure the high quality of my work by sharing information with the rest of the team. For example, I create design documents and specifications when developing a new function in order to share and engage in quality checks before going to the next step. To facilitate this process, I have developed an automated testing system to maintain the high quality of software products using less manpower.
Experiences of failure and growth
Q What challenges would you like to tackle now?
I currently am working on a project to integrate the management function of physical servers into the OpenStack platform. We developed LINE’s in-house software to manage these physical servers few years ago. However, we started to use OpenStack to manage virtual servers, so we need to operate the individual software stack to manage the virtual servers and physical servers. This increases our development and operation costs for these servers because we always need to develop the same features in both OpenStack and in-house software to provide the same level of functionality to our end-users, and we need to maintain the individual control plane for them. In order to improve and simplify this, we decided to abandon current in-house software and migrate it into OpenStack. OpenStack has a physical server management feature, called Ironic, but the feature does not match LINE’s environment, so we simply developed a new Nova driver from scratch to manage our servers efficiently. And we chose Kubernetes as the base environment for the new control plane so that we could make the control plane distributed, elastic and resilient for physical server management.
Q Have you made any mistakes at LINE?
Since we operate a huge private cloud environment, we sometimes have large system outages which make the Verda environment completely unavailable for end users. One of our latest outages was caused by OpenStack Keystone going down. In OpenStack and Verda’s other services like Verda Kubernetes service (aka VKS), they rely on Keystone for user authentication. So in this situation, we need to consider: (1) Shutting down the service which is causing the issue, (2) Recovering other unrelated services, (3) Developing a workaround to re-enable the service, which could trigger the issue again, to minimize the downtime of the whole services. It took a long time to recover Keystone from that outage because we needed to solve the root cause of the issue. Looking back at this, we learned we always need to consider two directions for dealing with outages, (1) Minimizing the impact on users, (2) Solving the root cause of the outage. In Verda, we have lookback meeting for every outage and make targets for improvements. This positive loop makes our platform stronger.
Wrap-Up
Q What are the goals you want to achieve at LINE going forward as well as your future career path?
I would like to keep developing myself as an engineer. In the medium-to-long run, I hope to complete various missions as Tech Lead, working with competent engineers. LINE has plenty of interesting challenges, so I will be able to have a lot of exciting experiences going forward.
Q Lastly, what kind of person would you recommend LINE to as a workplace?
In our industry, new things like new software functions and trends come up on a daily basis. I’d recommend LINE to those who are inspired by such new things or who are excited about creating something new, looking into and developing it – in other words, those who want to enjoy the life of a software engineer. If you’re the kind of person who can achieve team goals while maintaining a genuine interest in your own work, I’d like to have you on my team.
