As ITTT has been providing high quality TESOL courses in many different formats since 1998, we are, without a doubt, one of the market leaders in this specialized field of teacher training. Despite this extensive experience, we do not rest on our laurels, but instead we're continuously improving the overall quality of our courses and the services we provide. In the years since we've developed our first TESOL courses, tens of thousands of trainees have graduated with us and gone on to have amazing adventures teaching English in countries all over the world. Since we first started out providing TESOL certification, the English language teaching world has moved on at a rapid pace and we've kept abreast of that pace by continually developing new and improved courses, materials and methods of study. Our current range of courses and services we provide are certainly the most comprehensive and user-friendly we have offered to date. But to keep up with an ever-changing world, we continue to evolve in order to give every trainee the very best start to their new teaching career. As we have been helping potential teachers to get TESOL qualified and ready to head overseas to teach English for many years, employers around the world are well aware of our courses and certification. In fact, many employers actively seek out our TESOL graduates because they know they will have completed a highly respected course that has prepared them for life in an international teaching environment. This recognition applies to all regions of the world as our long list of graduates, going back many years, have taught in almost every country you can think of. In years gone by, it was possible to jump on a plane to almost any destination and find work as an English teacher without having any previous experience or qualifications. However, those days are now long gone and a reputable TESOL certification is seen as mandatory in most countries, particularly those that offer the best paid jobs and comfortable working conditions. If you're looking for an overseas teaching adventure that will also pay off financially, a highly respected TESOL certificate from ITTT will help you to stand out from the crowd. All ITTT online TESOL certification courses are accredited by leading institutions in the world of English language teaching, including the online TESOL and TEFL standards agency OTTSA. OTTSA is an educational organization whose main goal is to increase the overall standard of the international teacher training industry. Within this remit, it specifically concentrates on online based TESOL certification courses. Through OTTSA, our online TESOL courses have been evaluated and have successfully met all the criteria necessary to be accredited by this prominent and highly respected organization. Accreditation from leading institutions such as OTTSA is just one of many reasons why ITTT online courses are among the most popular in the TESOL world. Knowing that your online certification meets all the professional standards set by an independent body means, you can get on with your studies with confidence and look forward to a bright and exciting future as a TESOL qualified teacher. One of the major advantages we can offer our course trainees is the fact that all of our staff members have experienced what it's like to take a TESOL certification course and to live and work as an English language teacher overseas. From our in-class teacher trainers and online tutors to our booking office and post-course support team, we have all been right where you are now. Initially, we know it can be difficult to choose the right course to suit your future plans, which is why we are happy to offer all the help and advice we can before you make a final decision. Once your course is underway, our highly experienced and qualified in-class trainers and online tutors will be there to guide you through the materials, to ensure you fulfill your potential, and reach all of your study goals. Having been through the same challenges, every member of ITTT staff knows the best way to move forward in any given situation, so you can rest assured you will always be in good hands. Although your ultimate success as an English teacher will be down to your own hard work, it surely can't hurt to have an experienced and dedicated team on your side from day one. At ITTT, we put a strong emphasis on customer care. As we know that getting all the information you need and then choosing the right course to suit you can sometimes be a difficult task. If you have any questions regarding our TESOL courses or have any other pre-course queries, you can contact us via a wide range of convenient platforms. Our online contact form is a simple and effective way of sending your questions in and we will do our best to get back to you promptly with all the answers you need. You can also chat to one of our friendly and knowledgeable staff members via WhatsApp or Facebook Messenger. On our contact page, you will find several phone numbers that you can contact us on directly, depending on your location. You can even arrange a call back from us at a specific time of day over the phone or via a range of online services. No matter how big or small your question, we will be happy to help you, as you're unlikely to be the first person to have asked it. No matter where you are in the world, our international 24-7 customer care service is always on hand to take the stress out of your TESOL certification. It's to taking a TESOL course in an in-class environment and the biggest is probably the real-life teaching practice that makes up a large part of the four-week schedule. Throughout the course, each trainee will have many opportunities to practice the skills they have learned in the input sessions by teaching their own lessons to a group of genuine English language learners in an authentic classroom setting. What better way to prepare for the realities of teaching than having multiple practice sessions under the guidance of one of our professional teacher trainers. In preparation for each practice lesson, you will receive plenty of help and advice during the planning stage from our team of trainers, who will then follow up the lesson with a debriefing session where they will highlight the areas where you were strong and any areas that you could try to improve on for future lessons. By completing the in-class TESOL course and all the teaching practice that comes with it, you should be perfectly placed to secure a good teaching job. If you want to find out more about TESOL courses and teaching English abroad before you sign up for a course, check out our free downloadable information packs. The ITTT brochure is a great place to start as it contains a wealth of information regarding our wide range of training courses so you can make an informed decision about which one is the right fit for you and your future plans. Is an in-class TESOL course in a tropical location the best way for you to start your overseas teaching adventure? Or would an online training course suit you better? Download the free brochure now to find out more. Once you have swatted up on our various TESOL course options, you should then take a look at our free e-guide to TESOL, which covers the wider world of teaching English abroad. From the basic requirements of teaching abroad to the best methods of finding, applying and interviewing for jobs around the world, you'll make sure you know where you stand. You will also find a handy guide to all the main countries where TESOL teachers are in demand. Under each country segment, there's a brief overview of the local job prospects, as well as an estimate of the average salary you can expect to earn. So what are you waiting for? Download the e-guide to TESOL and take the first step towards a new career in teaching English abroad. If, like many people, the idea of an end-of-course exam fills you with dread, you have nothing to fear with our online TESOL certification courses. All our online courses include ongoing assessments, as you work through the materials. Removing the need for a final exam at the end. Each course is divided into separate units with a multiple-choice assessment to conclude each section. Once complete, you then move on to the following course unit. There is no need to hit a required pass rate on each individual assessment either, but rather an average requirement of 75% across all the course unit assessments. Provided you reach this realistic assessment level, you will graduate from the course and receive your TESOL course certificate. Thanks to this user-friendly approach to assessment, and the flexible nature of our online TESOL courses, trainees should be able to work through their studies with minimal levels of stress. Instead of worrying about an end-of-course exam, you are able to comfortably work through the course and spend a bit more time planning your overseas adventure. Nobody likes small print, as it can catch you unawares and cost you additional money that you had not bargained for. At ITTT, we do away with all that nonsense and show you the full cost of every online TESOL certification course in plain sight, right on our website. Whichever online course you choose, you can rest assured that the advertised course fee is all you will have to pay as we have no hidden costs to worry about. The course fee advertised will include everything you need to complete your training and get you ready to start teaching English wherever you choose. The advertised fee for our online TESOL courses includes all the written materials and video access that is relevant to your course, as well as the high-quality certificate that you receive once you have completed your training. We even cover the cost of posting your certificate to you wherever you are in the world. The one and only additional costs is if you choose to have your certificate sent by Express Delivery which has a small fixed fee to any location. Other than this optional extra which is unnecessary for most graduates, you will have nothing else to worry about once the initial course fee is taken care of. Do you want to start a new career teaching English abroad or are you concerned that you lack experience and qualifications or are simply too old? Don't be. As far as you are concerned, the international market for TESOL qualified teachers is so vast that there are plenty of great opportunities out there for everyone. Previous teaching experience is certainly not an issue as the vast majority of people teaching English overseas have never worked in a classroom before they took up their existing post. Having a degree is also not essential to your potential success in the field of English language teaching. While there are a few countries where government restrictions mean that only degree holders are eligible for jobs, they are far more that have no such regulations. Many potential teachers also worry their age might go against them when looking for a TESOL related job. Once again, there are a few countries that have a mandatory retirement age for teachers but these are far outnumbered by those that are happy to employ teachers regardless of the age in their passport. At ITTT we have trained many thousands of teachers over 50 years old and beyond who have gone on to have great success in classrooms around the world. Regardless of your background, academic achievements or age, you are welcome to join any of our TESOL courses because we know that every fluent English speaker has the potential to become a great teacher. As the market for teaching English abroad is so big there are a huge range of opportunities out there for people of all different nationalities and backgrounds. In many countries, there are multiple job openings available for every TESOL qualified teacher which is one reason why non-native English speakers generally have no problems in the international job market. In fact, of the many thousands of teachers that graduate from our courses every year there are more non-native English speakers than there are native speakers. No matter where you were born or what your native language as long as you are fluent in English there is no reason why you cannot go on to have a long and successful career in teaching English. Non-native English speakers will also find that they enjoy certain advantages over native speakers. In many cases, you will likely have a better understanding of the mechanics of English as you have had to learn it all from scratch. Many students of English also find it easier to learn from a non-native speaker as they can often have a greater empathy with the learners in their classroom due to their own previous experiences. The bottom line is being born in a non-English speaking country should be no barrier to your dreams of teaching English abroad or in your home country. One of the major attractions of our online TESOL certification courses is the flexibility they provide. As each course has no set study times or specific targets you are free to set your own study timetable to fit into your own unique set of circumstances. If you have existing work, study or family commitments, there is no problem as you can study as much or as little as you like whenever you have a free slot in your schedule. There are no set times when you have to be online or in the scheduled online classes that you need to attend. Simply log into your personal online dashboard anytime you like and spend whatever free time you have available on the course. To ensure that the information covered in the early parts of the course is still fresh in your mind once you have worked through all the materials there is a 6 month maximum time limit from the day you get started. Within this generous timeframe you are under no pressure whatsoever. It is entirely up to you how quickly or slowly you study. To ensure you are able to access your online T-SOL course dashboard whenever it suits you it is available on any device you have to hand. You can log in via a PC, laptop, tablet or smartphone from any location you choose as long as you have access to the internet. This high level of flexibility allows you to take the most of any spare time you might have 7 days a week. If you are away from the internet for any reason you can also download the course materials to your device so you can access them whenever you need them. You might even want to print them off if you prefer to study the old fashioned way. Our online T-SOL courses also come with a range of teaching videos covering many subjects that are included in the written materials. These videos provide an extra insight into each subject and can really help to reinforce each individual course topic. Once again for ease of use these videos can be downloaded onto any device so you can watch them as many times as you need in any situation. By choosing an online T-SOL certification course you are in no way compromising on the theory of English language teaching as they cover a wide range of essential topics. Our most popular course the 120 hour T-SOL certification includes in depth units on all the main subjects you need to know about before you start your first teaching job. A great deal of coverage is given to English grammar subjects and how to teach them in the classroom. These include the tense system conditionals, modals passive voice and much more. Initially these subjects can seem a little scary but there is nothing to fear as our high quality materials will guide you safely through each topic. Another very important part of teacher training is classroom management skills. Regardless of the age or nationality of your students your job as a teacher will be that much harder if you are unaware of the basic methods for organizing and controlling a classroom. Once again this is covered in detail during our online courses. All ITTT T-SOL courses come with extensive and in depth materials that are designed to gradually build your knowledge as you work through the course. Although the materials cover a wide range of key subjects they are put together in a way that is informative, interesting and easy to follow. So you should never feel bored or bogged down with too much technical information. By keeping our materials current and avoiding the bland dullness of some educational courses we aim to provide a learning experience that is rewarding and fulfilling. At ITTT we design our courses with the ultimate goal of ensuring graduates are fully equipped to enter the classroom on their first day with a spring in their step feeling ready and able to deliver a lesson that is both professional and enjoyable. To make this happen we constantly review our courses and materials to ensure they are up to date and in tune with the latest developments in the English language teaching world. Whichever course you choose you should find that it ticks all the boxes required by international employers and provides an enjoyable way to kick-start your journey teaching English abroad. Regardless of which online TESOL courses you choose you will have the option of taking a tutored or a non-tutored version. Unless you are particularly used to distance learning an online tutor is generally recommended as it provides you with access to a professional tutor whose one and only focus is to guide you towards successful completion of your training. No matter how big or small your problem or question your tutor will be there to provide all the answers and guidance you need to keep moving through the course. To ensure that our tutors are able to provide the assistance and expertise you need throughout your TESOL course we only employ highly experienced tutors who have extensive knowledge of life as an international English language teacher. They lived and worked overseas in a variety of different classroom environments and they possess a range of advanced teaching qualifications such as the diploma in TESOL. Having an online tutor to guide you through your studies is an extremely popular option as your tutor will know exactly what you are going through as they will have been in the exact same position themselves in the not too distant past. There are many great benefits to taking an online TESOL certification course and one of the most popular is the fact that it can be taken from absolutely any location in the world. As long as you have access to an internet connection and the device to run it on, you are in business. As all of the written course materials and instruction videos can be downloaded for off-land use you do not even need to have a continual internet access to successfully complete the training. Similarly, while some people have access to a powerful personal computer this is not a necessity as our online courses can be just as easily taken using a laptop, tablet or even a smartphone. How and when you choose to study is also in your own hands. To allow for maximum flexibility you get to set your own study hours as there are no formal lessons or times when you have to be online. This is also important when you consider that our trainees are spread all over the world in every different time zone. Although an in-class TESOL course is highly practical in nature at ITTT we ensure that our online courses also cover as much practical content as possible. Throughout each course we cover a wide range of theoretical subjects that are important in developing your teaching knowledge. However, we also do our best to make each subject come to life as you will experience it in your future teaching career. Specific teaching skills that are essential in creating a productive learning environment such as lesson planning classroom management and testing procedures are all covered in depth to ensure you have a rounded understanding of both the theory and the hands on nature of EFL teaching. To further enhance the written materials contained within our online courses we also include access to a number of practical videos that should help to give you a better understanding of how to deliver your lessons to the live classroom. These videos are designed to work alongside the written materials to further enhance each subject. For even more free TESOL related content, check out our YouTube channel where you will find over 2,000 videos covering a wide range of subjects. You will find videos looking at different TESOL courses and course reviews from some of our previous graduates. There are also many videos that provide teaching ideas for different areas such as language grammar and videos that cover essential teaching skills such as lesson planning, teaching methodologies and classroom management. We also look at some of the most popular teaching locations and provide tips on getting the most out of your new home away from home. Our range of insightful videos is growing all the time adding a visual angle to all our other forms of information for those thinking about taking a TESOL course in the midst of doing a course or graduates looking to find their first job as an English teacher abroad. Whatever your current situation, take a look at our videos page because you might find just what you need to take the next step forward. And don't forget to check back often as the world of TESOL never stands still and neither does ITTT. In recognition of the hard work you put into our TESOL courses every course graduate will receive a high quality, embossed certificate as part of the overall course fee. Each certificate we issue includes a unique certificate number which can be used by potential employers or government departments to verify that it is authentic and you are indeed a fully certified ITTT TESOL graduate. Anyone who wants to verify your certificate can go to the certificate verification page on our website. Enter the required details and an instant result on its authenticity. This is an increasingly useful tool for international employers as it provides a quick and easy way to catch out the small minority of cheaters who try to get by on a fake or copied certificate. With a genuine certificate from a well respected course provider, you can be confident that all your hard work will be more than worth it. No matter where you want to live and work as an English language teacher, your TESOL certificate from ITTT will be your passport to the very best teaching jobs worldwide. As soon as you have completed your TESOL course, your high quality certificate will be put in the post and sent to wherever in the world you want it. However, if you are in a real rush and you don't want to wait for the postman to arrive, we can also send you an instant copy in PDF format via email. This is a handy option for anyone who needs to prove they have completed the course to an employer or government office before the original certificate arrives. As our certificates are sent to all corners of the globe, it is not always easy to predict how long each postal delivery will take. If you think you might need your certificate soon after finishing your training, simply let us know and we will immediately send out an electronic copy. And don't forget the postage cost of your original TESOL certificate is covered in the course fee to any worldwide location, and the PDF copy is also free of charge. Whatever your teaching plans, there is no need to wait around with ITTT. Although every course graduate will have their TESOL certificate sent in the post as soon as they have completed all sections of their course, we cannot guarantee how long the delivery might take to different destinations. For most people, this is not a problem, but if you are on a tight schedule and really want that certificate in your hand as soon as possible, we can arrange an express delivery option via DHL. This optional extra is available for a small fee to any location across the world. For most of our course graduates, this extra expense is not necessary. But if you need to prove your qualification to a potential employer or government office and don't want to wait, it could be money well spent. Alternatively, a temporary electronic copy of your TESOL certificate might be sufficient in many cases and we can arrange this for free in an instant. Simply let us know if you need either of these alternative options and we will make the arrangements there and then. If you want to get your TEFL journey started in a hurry, you can rely on us to make it happen. If for an employer, government office, or just for your own peace of mind, you require additional authentication with your TESOL certificate, we have several options to choose from. One option is via OTTSA who moderate and validate our courses. For a small fee of 25 US dollars, they will stamp your certificate with an additional seal of approval. Another popular option is to have a notarized copy of your certificate printed, stamped, and signed by a Thai notary public as Thailand is where all our certificates are printed and posted from. This official copy can then be sent to you along with your original TESOL certificate. The fee for the notary public service is 45 US dollars. A third option is to have a copy of your original TESOL certificate authenticated by the ITTT registrar and then have it counter signed by a notary public at the same time. Once again both the original certificate and the verified copy can then be sent to you anywhere in the world by post. The fee for the registrar verification and notary public authentication is 95 US dollars. Although most of our graduates do not require these services, we are on hand to get them sorted out if and when required. As an ITTT TESOL course graduate, you are able to issue a high quality certificate of completion to any English language student who completes a series of lessons with you. Each certificate allows you to enter the name of the student, the level of language class they completed and how many hours of study the course involved. Every certificate you issue has a unique identification number and a box where you can enter your own ITTT certification number.  Being able to offer an official certificate of completion to your students means you can potentially stand out from the competition if you are teaching in a crowded and competitive market. They can also provide additional motivation for students in the classroom as they will have something to show for the effort they put into their studies. Essentially having these unique course completion certificates in your toolbox adds one more string to your bow as a professional TESOL qualified teacher. If you choose to attend an in-class TESOL course at one of our many international training centers, the certificate you receive at the end of the course will be validated by Lamar University, Texas. Lamar University is part of the Texas State University system and is accredited by the Southern Association of Colleges and Schools Commission on Colleges toward degrees at the associate, bachelor's, master's and doctoral levels. Through Lamar University, your TESOL course will be seen as the equivalent of a three-credit undergraduate university course. An official transcript for the course is also available for a small additional fee should you require it. This additional validation is another reason why you can always rely on our TESOL certificates to set you on the right path towards a rewarding and successful career teaching English at home or overseas. Thousands of ordinary people just like you have already done it. So what is stopping you? As part of our commitment to our trainees, we do all we can to ensure you have the best chance of finding a suitable teaching job after you graduate from your chosen TESOL certification course. To help make this happen, we have a wide range of post-course services available to anyone who requires them. We can offer practical help with constructing an effective CV or resume and cover letter as well as provide leads on potential jobs around the world. Every course graduate also gets exclusive access to our own database of employers, many of which are always keen to recruit from our alumni as they know and trust the quality of our teacher training. Once you have tracked down the job or jobs that you want to go for, we can help out with advice and guidance on preparing for an interview and a trial lesson if the employer requires one. We can also take job contracts you are offered to ensure you are getting a reasonable deal and are not signing up for a less than reputable position. And the best news is that all this assistance is available to every one of our course graduates for as long as you remain in the world of English language teaching. By starting your TESOL journey with us, you get a lifetime commitment to job assistance that could take you to places you never even knew existed. Although we pride ourselves on providing the best TESOL certification courses on the market, we also do all we can to ensure our graduates have the best choice of job prospects once they have completed their training. Our lifetime job assistance program includes a wide range of great tools to help you find the perfect job in the right location for you. One of these tools is the job postings we provide across our websites and social media platforms. Our job listings are updated every single day, so you will always be one step ahead of the competition when it comes to landing your dream job. As a long-established TESOL course provider ITTT has made many contacts with employers across the world that are keen to employ our graduates as they know they will be knowledgeable and ready to teach their own classes from day one. Thanks to these contacts, we often have unique job listings that you won't find on any other website. With a TESOL certificate from ITTT, you don't just get the qualification you need to start teaching English abroad, you also get access to potential job opportunities from all corners of the globe on a daily basis. As an extended part of our lifetime job assistance program, we also maintain close partnerships with several of the top recruitment companies operating within the TESOL world. Through these long-established and well-respected companies, you can find teaching jobs around the world without much of the hassle associated with applying for jobs directly. By using a recruitment company, you can typically arrange a teaching position before leaving home, which can be a real comfort, particularly for those who are new to teaching and traveling abroad. Job applications and interviews are generally done over the internet. And paperwork, such as visas and work permits, are usually taken care of for you. Another big benefit of using a recruitment company is that it won't cost you a single cent, as it is down to the employer to pay for the recruiter's services. You can also register with more than one company to give yourself a wider choice of potential jobs to choose from. Once you have completed your TESOL course and are ready to get going, a recruitment company could be the best option for you, if you want a straightforward and efficient route into teaching English abroad. At ITTT, we have many ways to help you find your dream job, wherever in the world it might be. At ITTT, we don't just sell you a TESOL training course. We also guide you right through the entire process from start to finish. Initially, we can help you decide which training course is the right one to suit your future plans and give you advice on how to approach your studies. Once your studies are underway, we will be right by your side as every course comes with the option of an online tutor, whose main focus is to ensure you successfully complete the course.  As soon as you have completed your training and are officially TESOL certified, you will no doubt be looking to lend that dream job somewhere around the world. Luckily, our guidance doesn't stop once we issue your certificate. As we also provide an exclusive job assistance program that provides a whole range of help with finding and securing that all important first teaching position. As if that was not enough, our job assistance program is available to you for as long as you continue your career in the field of TESOL. Whenever you feel like a new challenge or simply want to head off to a new destination, we will be there to offer all the help we can. The world of TESOL branches out into several specialized areas that require a unique set of skills and specific knowledge on the part of the teacher. To ensure all of our course graduates can access these potentially lucrative markets, we have developed a range of specialized TESOL courses that are designed to build on the content of a standard TESOL certification course. Our courses in teaching young learners and teaching business English provide all you need to expand your international job options to these two major areas of teaching English abroad, as well as giving you extra knowledge and skills, they also look great on your CV or resume. Another area of TESOL that is rapidly expanding is teaching English online. This option is particularly popular with teachers who are unable to head overseas due to unavoidable commitments and would like the opportunity to work from home. It can also be a great way to gain initial teaching experience before setting off to teaching English abroad. There are hundreds of thousands of potential students worldwide and many different platforms and agencies that you can use to get your online teaching career off the ground. Would you like to work from home? Be your own boss and earn good money. If you are a native English speaker or you're fluent in English, now is the perfect time to earn a living working from home. Get TESOL certified and start teaching English online from anywhere you like. And when you are ready to travel, you can also teach in countries all around the globe living and working abroad. Get started today. Visit www.teflcore.net Apply to get TEFL or TESOL certified and start your new life. By choosing an in-class TESOL course, you will have the option of gaining your qualification in any one of a number of amazing locations around the world. Whether you prefer to visit a major world city, such as Bangkok, Barcelona or Beijing or a popular tourist destination such as Gran Canaria, Manuel Antonio or Phuket ITTT has got it covered. There are in-class TESOL course locations in Africa, Asia, Europe, North America and South America. So wherever you live, there is a good chance there is a training center in the region. Of course, not everyone chooses to take an in-class TESOL course in their own backyard. Many people take the opportunity to visit a destination they have always wanted to go to and so kill two birds with one stone. Another bonus of taking an in-class course option is that all our training centers are situated in areas where the demand for qualified English language teachers is particularly high which makes your chances of securing a good job after the course that much higher. Our training staff can also help the job search as they will have an in-depth knowledge of the local job market. Whichever course location grabs your interest, you are certain to have the adventure of a lifetime. In-class TESOL courses have many highlights such as in-person instruction from professional teacher trainers and invaluable teaching practice with real English language students. On top of these major benefits, there are also a few extra bonuses on offer at some of our training districts. In many cases, our training staff are able to arrange free airport pickups which can be a great help if you are visiting a new country for the first time. The knowledge that you have transport to your accommodation arranged in advance means you can sit back and relax during your journey without worrying about finding a taxi, bus or train on arrival. Although the in-class course does involve some hard work, there will still be plenty of time during the evenings and weekends. Most of our training centers organize a welcome dinner at the start of each course where trainees and course trainers get to meet each other and to enjoy some local food and drinks in a fun and relaxed atmosphere. The combined TESOL course provides a unique blend of online and in-class study that is perfect for some trainees. Once you have completed the online component of the course, you then get to attend one of our training centers for a limited time in order to gain some invaluable teaching practice in a real language learning environment, as well as further instruction from our professional teacher trainers. If you choose the combined TESOL course option, you get to pick from a number of great training center locations. The in-class section of the combined TESOL course consists of 10 days of study with the middle weekend free. If you choose our centers in Bangkok, Phuket or Rome, at our centers in New York and Paris, the in-class section lasts for 8 days straight. With these great locations to choose from, many trainees take advantage of their visit and incorporate a bit of extra time to enjoy the local sites and attractions. The combined course will set you up so you're ready to take on any challenge that your new teaching career has in store for you and give you the chance to explore one of the world's most popular destinations. If you are currently living in America or Canada and would like to get started with your new English language teaching career, you can take a short form combined course that features convenient online study followed by 5 days attendance at one of our training centers. The main benefit of this option is you get face-to-face instruction from a professional teacher trainer to back up all the theoretical knowledge you have already covered during the online section of the course. This convenient TESOL course option is currently available in 16 great locations across 13 different states as well as in Toronto, Canada. You might want to visit one of the country's major cities such as Atlanta, Los Angeles, New York, Orlando, San Diego or San Francisco while smaller locations including in Arbor, Columbus and Madison are equally popular. Each location has its own attractions but they all share the same high level of teacher training in a professional environment. If you just cannot wait to get your teaching career underway but you also want the benefit of some in-person training, the 5 day combined course could be just what you're looking for. There are many benefits to taking a TESOL course in an in-class environment and the biggest is probably the real life teaching practice that makes up a large part of the 4 week schedule. Throughout the course, each trainee has many opportunities to practice the skills they have learned in the input sessions by teaching their own lessons to a group of genuine English language learners in an authentic classroom setting. What better way to prepare for the realities of teaching than having multiple practice sessions under the guidance of one of our professional teacher trainers. In preparation for each practice lesson you will receive plenty of help and advice during the planning stage with the trainers. Who will then follow up the lesson with a debriefing session where they will highlight the areas where you were strong and any areas that you could try to improve on for future lessons. By completing the in-class TESOL course and all the teaching practice that comes with it you should be perfectly placed to secure a good teaching job and have the confidence to jump straight into your own classroom and a wide choice of high-quality TESOL courses we also believe in providing excellent value for money. To ensure the world of teaching English abroad is open to people from all walks of life we offer TESOL certification courses to suit all budgets. From our 60-hour introductory course to our 250-hour diploma in TESOL you should find a course that is affordable and suitable for your future employment plans. Keeping the cost of our courses as low as possible we hope to give everyone the chance to achieve their dreams of a career in English teaching. We also offer further value to our trainees by giving away free additional courses when you sign up for one of our most popular options. Many of our courses come with a free specialized course in teaching young learners or teaching business English which will add a real shine to our students. Whether you want to volunteer in a primary school in Uganda or earn big bucks in the oil-originations of the Middle East you can do it with a great value TESOL certificate course from ITTT. In order to make your TESOL training experience as smooth as possible we are able to accept a wide range of payment methods. If you want to get started as quickly as possible we recommend using a credit or debit card which is a quick secure and hassle free option via our online payment page. As soon as you make the payment you will be able to get your exciting new life as a TESOL qualified teacher up and running. We also accept other quick and easy online payment methods such as PayPal and WeChat which also allow for a quick and convenient way to get the ball rolling. If credit cards and online payments are not your thing no problem. We also accept additional payment methods including Bank Transfer and Western Union although these options might take a few days to clear they are still a tried and tested way of making a course payment. Finally make sure you check out the discount page on our website before making a payment as you might find you can get some money off your course or get an additional course free of charge. At ITTT we do all we can to make your TESOL experience both convenient and great value for money. As part of our commitment to providing the best value for money TESOL certification on the market we regularly have a range of special offers available to our trainees. One of our regular offers is the option of a free additional course if you sign up for an in-class, combined, or 120 hour online TESOL certificate course with tutor support. With this great offer you can choose one of our specialized TESOL courses or the diploma in TESOL as your free course which is an ideal way to widen your job prospects and enhance your CV or resume without parting with a single cent. We also offer significant discounts when two or more people sign up for a course together. If you have friends or work colleagues who share the same dream of traveling and working as an English teacher abroad you can all save extra cash for your trip by starting a special journey at the same time. Before you enroll on any of our TESOL courses check out the special offers page on our website to take advantage of any discounts or free courses that could make a real difference to your bank balance and your future job prospects. Online courses allow trainees to study at their own pace from anywhere in the world which is why the majority of ITTT trainees prefer to take the online format. While in-class courses were preferred in the early stages of EFL teaching, online courses have become the standard certificates on the EFL market today. All ITTT online courses are accredited and externally moderated by OTTSA, the online TEFL and TESOL standards agency. You can find more information about our accreditation on our website. Look at the top of every page for the accreditation tab. ITTT is one of the largest TEFL and TESOL providers in the world and has been in business for over 20 years. We have over 107,000 course graduates, most of whom are currently working as English language teachers all around the world. Our 120-hour TEFL and TESOL certificate is considered the international standard for teaching English abroad today. On top of that by completing our more advanced courses, such as the 220-hour master package, the 250-hour TESOL diploma or the 470-hour professional package, you can further increase your chances of landing a top position teaching EFL. We also recommend checking out our Facebook page to connect with fellow trainees and to read through some detailed reviews of our courses. You can find our Facebook page through our website or by going to www.facebook.com slash tesol.tefl. Please visit www.teflcourse.net to apply for your course. We look forward to helping you teach English as a foreign language in the destination of your choice. Our Facebook page provides an important hub where our students and course graduates can connect with each other and share advice, tips and information on all aspects of TESOL. One of its many benefits is that it is made up of people from all nationalities and backgrounds, so everyone should be able to find what they are looking for. Feel free to join our online community and put your questions, recommendations and course reviews out there to not only find the answers that you are looking for, but also to help others like you to make the right choices to suit their situation and future teaching plans. Our Facebook page is also a great place to access other content such as our web page, YouTube channel, special offers, photos and much more. We currently have well over 100,000 followers and the numbers are rising rapidly thanks to our fresh and insightful content that is provided by our TESOL experts, current trainees and graduates at all different stages of their teaching adventure. Why don't you head over to Facebook right now and see what all the fuss is about because they can't all be wrong. While we have always tried to put the needs of our trainees first, it is good to get feedback from our TESOL course graduates once they have earned their certification to hear about their personal experience of training with ITTT. In the vast majority of cases we are proud that our feedback is very positive and the ratings we receive are among the highest in the field of TESOL course providers. Of course, you shouldn't simply take our word for it. You can go online to any one of a number of trusted third party sites and see what people are seeing about us with your own eyes. For an insight into what our trainees and course graduates think about our TESOL courses and the services we provide, head over to sites such as Facebook, GoAbroad.com, GoOverseas.com, Reviews.co.uk, Teflereviews.com and TrustPilot for independent reviews from a wide range of people from all over the world. Although we are confident that our TESOL courses are the best on the market it is good to note that thousands of independent reviewers agree and are happy to recommend our courses right across the online community. We are proud to have been recognized within the TESOL world for the quality of our training courses and our user friendly service. Since 1998 we have received many awards from independent sectors within the teaching and travel industry including several recent awards. GoAbroad is one of the largest providers of overseas experiences online so we were honored to be presented with their award for top TESOL certification organization 2018. The icing on the cake was to then receive their award for top rated TESOL organization 2019. GoOverseas is another major independent source of online information regarding all forms of travel and adventure. Once again we were honored to be presented with their Community Choice Award 2018. Not only do these prestigious awards show that we are at the forefront of TESOL certification they also encourage us to further improve our products and services in order to maintain our position at the head of the teacher training field. Thanks to these many other awards from within the TESOL world you can rest assured that your overseas adventure is in safe hands with ITTT. If you want a one stop shop for all things TESOL related you should head over to the ITTT blog where you will find a vast amount of interesting information. The blog contains all manner of fascinating posts from professional TESOL experts and our very own course graduates. You will find helpful tips on visas at work permits, salaries and budgets and general TESOL course information. There is also a section on teaching locations that looks at many of the most popular destinations from a wide variety of angles as well as some great teaching ideas that you can use to transform your own classroom into a modern, fun and productive environment. A large part of the blog is given over to our TESOL course graduates at the time to write about their experiences of teaching English abroad both in and out of the classroom. If you are serious about heading overseas to teach, the blog can offer plenty of advice, tips and insight information that could make your journey a much smoother process and point you towards information you didn't even know that you needed as the blog is regularly updated with fresh and insightful content make sure you head there often to keep abreast of the latest insights into the current state of English language teaching worldwide. You might even want to add your own story to help inspire other potential teachers to take the leap.


No So I'm here to talk about current TLS and hardware TLS offload in free BST 13 This has been a long time effort. It started initially with Netflix and now also Melanox and Chelsea is involved and My name is like said Hans Peter Salaski also called HPS Or H. Salaski at free BST.org. I started out a long time ago with USB and now I'm doing full-time networking with Melanox and Here we have Drew Gallatin and I started off a long time ago with free BST alpha and networking and various things and Here we are with TLS. Why don't you go ahead Hans? Sure. So I Have a little petition here So how many has ever been to a cryptography course before at the university put your hands up So, yeah, there's a couple of people here So that's good. So you may be familiar with Bob and Alice. I need to say a little bit about why we do cryptography So Bob and Alice are the two famous characters that try to exchange a secret message And there's also the evil guy that tried to eavesdrop and do data tampering so Quickly summarized cryptography is making a numerical message depending on a small pre-shared key and You can use it right or you can use it wrong and Usually it's good. It prevents leaking data to others and also you can use it for check summing and Unfortunately also Cryptography can make your data disappear faster when you lose your keys So be careful TLS That's maybe familiar to a lot of you guys. It's short for transport layer security I'm trying to be very easy starting in the morning it's used behind HTTPS on port 443 and It can support multiple crypto codecs Like AES. We're mostly going to focus on AES because that's the main standard with TLS 1.3 and It can also support different key exchange protocols. So we you maybe know about Diffie Hellman and RSA and There might be more that we don't know about yet so like mentioned TLS is a protocol and It runs on top of TCP and Then on top of TLS you can run other protocols like shown in the slide and This is just to give you a picture and I'm going to dig a little bit into details When I started doing TLS work. This was a black box. What is TLS and trying to find Documentation was not so easy. So I started with the code TLS has a small header that encapsulate the data We have 13 bytes Typically, but it's variable So in the beginning we have a type It can be data handshake alert For example alert happens when someone is tampering with the data on the receiver Identifies oh, I cannot decrypt this packet in a sense alert message to terminate the connection Then you have some major and minor numbers and and for TLS 1.2 It's three and three This actually dates back to the times of SSL version three so so it's not actually should be one and two, but it's three and three and And the TLS length is a 16-bit number so you can actually encapsulate up to 64 kilobytes minus one to be exact but a lot of applications limit this to 16 kilobytes and and Yeah, that's just a kind of legacy thing You need to know that when you use TLS you might need to use smaller blocks You cannot do so big blocks After the length you have some variable nonce. It's usually eight bytes with some Crypto codecs depending on algorithm and sometimes it's not present And then you have the dot on So this is TLS 1.2 So the difference for TLS 1.3 is basically that you don't have a nonce The nonce is kept on the sidetrack of the protocol. It's maintained by the Hardware or software and to save bandwidth and As you can see here the major number and minor numbers are still three and three So why is that? That's again We have a lot of routers and equipment that only support TLS 1.2 And if you try to change these numbers Then maybe that equipment won't support this protocol and your packets will be dropped so in instead of putting the First byte the TLS type in the beginning. We now put it after the TLS dot. That's not shown in the picture and Yeah, this is just a curiosity at the moment I'm going to say a few words about AES AES is a old algorithm relatively old Started in the Netherlands and it wasn't called AES You can read about it at Wikipedia it's basically doing 16 and 16 bytes at a time and It can be used as a stream Version that means you can stop Encrypting and keeping the state and you can resume encrypting So basically you can When you're encrypting a stream you can do byte by byte Because it used the previous block Output from the previous block to encrypt the next block and in the beginning there is Initial vector that you use to start with And I can also mention free BST supports the non-stream version of AES too The transport layer of security is in free BST implemented by open SSL I know there are other alternatives like Libre Libre SSL But I'm going to focus on what we have in free BST at the moment You may be familiar with the term AES and I that's AES new instruction set It's usually a CPU offload for AES And it makes it run faster Then then we have also a software kernel TLS That means instead of doing this encryption in the user space We do it in the kernel instead and I will return to why we do that later on in the presentation There's also something called open crypto framework or OCF that is basically PCI card that where you can DMA the data and get the encrypted data back and that's for the kernel and Then we have yet another technology. It's called TCP offload engine or TOE That means we send only the TCP data to the network card and then a network card will do both the TCP and the TLS in the same operation and Then we have Nick kernel TLS That is when we're sending full TCP frames with data to the Nick and the Nick will then decipher the header and Undo the TLS encryption Packet by packet and this way you can also do TSO So so you can put down a big chunk of data to the Nick And then Nick will do both the fragmenting of the frames and it will do encryption at the same time So I'm going to look a little bit in open SSL Open SSL is you can look at it like a filter It's based around something called a bio structure It's like a source and sync for data and you can hook them together to To make a chain of filters like you can read from a file. You can do encryption and can output to a socket and With this framework all data must have a pointer and user space So so it's passing around pointers It's zero copy inside open SSL But when you do a socket to send then it will be a copy into the kernel So I will talk more about this later on as well Open SSL and kernel TLS so We have a guy at my office in not in Norway, but in Israel That's called Boris. He made 16 patches to support something called kernel TLS. It's like offload for doing the encryption in the kernel instead of in user space and He did it initially for Linux and Now we also have this in free BST So the API is very simple It's to set socket options. You have set such a set socket option that turns on TLS for TX and you have a set socket option where you can switch the back end you are using in the kernel So so you can for example say I want to use the nick offload I want to use open crypto framework. I want to use something else So so you can switch around which back end you're using and I have a link here which shows when free BST support was added It's revision three five one five two two and this is a cumulative work of many people So John Baldwin sitting here on the front did push button work and Get it got it into the tree, but but it's really Like Drew here will mention in this part. It's it's it's a lot of people involved Okay, as Hans was saying a lot of people were involved in this back in 2014 2015 Netflix made a commitment to protect the privacy of its users and to start encrypting the via TLS the streams that We send the movies to your to your devices in and so the problem was this is really expensive. How are we going to do this and Scott Long and Randall Stewart did Scott long have the idea Randall Stewart did the initial implementation of Colonel TLS and the idea was to Preserve our normal Sun file pipeline where you know instead of what what a lot of people do Which is to read the data in you know from the kernel into a web server And then and then write the data from the web server back into the kernel We kind of want to avoid that extra step and we just want to be able to do a send file So by doing TLS in the kernel we can preserve that same Pipeline and we can still use a async send file and everything looks basically the same except for the crypto step And the idea was we want to do it as you know as as efficiently as possible and a huge amount of time was spent by Randall Making it efficient and then even more time was spent by me coming along after Randall and even making it even more more efficient so in order to do some of these things in for TLS we needed some enhancements to M buffs and I'm going to talk about the not ready flag I'm going to talk about the and I'm going to talk about unmapped M buffs and then Hans is going to continue on with syntax for Nick TLS So what's the not ready flag? This is something that Gleb Smirnoff also from Netflix came up with to support async send file and the idea is that When you are doing send file you're reading stuff from disk and when you whenever you read stuff from disk There's a chance you're going to block So rather than having nginx have to block and lose an nginx context and have to have a thread pool The idea is that nginx Uses async send file. So what happens is you send file submits the The unbuff into the socket buffer and it issues a disk read to fill the pages that are attached to the unbuff But when it puts the unbuff in the socket buffer it marks it not ready And what that means is when TCP is processing the socket buffer looking for things to send it has to stop when it runs into But not ready. So when the disk interrupt handler comes And the pages are now there it marks the unbuffs ready and calls the TCP ready routine which which then calls TCP to reexamine the socket buffer and Send anything which has been marked ready and in that way you can avoid having an nginx thread pool where you're having lots of context blocking the The handy thing is that I realized that it allows for a very simple way to sort of add a stage to that pipeline and After the the pages coming from this you can leave them not ready and call a crypto routine Which will then mark them not ready so that in that way we can use the not ready flag for kernel TLS as well and so the next thing I want to talk about is unmapped n buffs and what that really means is a An m buff that's basically pointing to an array of physical First to start off is physical pages and in fact the structure is still kind of named that way, but it's really just physical addresses So it was initially and I initially thought of it for send file and not for TLS and the idea was that in Send file you have one m buff pointing to one page So for you know 64k you've you've got 64k divided by by 4k or like 16 pages you so basically for For every 4k you're in a socket buffer. You're walking a new m buff you're taking a new cash miss and TCP walks the socket buffer chains a lot, especially you know processing acts and doing things like that If you can combine all of these all these into an array So instead of having just Instead of just having 4k reference to have 16k in the case in the case of TLS or you know like a hundred K in the case of Non-TLS you can reduce these cash misses a large factor and even in our unencrypted workloads At the time when I introduced this it reduced our CPU by something like between 5 and 20 percent depending on the on the on the machine So the other handy thing which I realized later is that it also provides a nice way to work with TLS So by enhancing this just a little bit by adding space at the front for the the 13 bytes That Hans was talking about for the beginning of the TLS record and adding some space at the back for the end of the TLS record All of a sudden you've got a single atomic way to refer to a TLS record And that's really handy for being able to do reference counting for TCP retransmits for Nick TLS and The reason that's important is because TLS records don't always end up lining up with TCP segment sizes, so what can happen is TCP can get an act for You know up to a certain point in a stream Whereas that might be in the middle of a TLS record So what TCP wants to do is TCP says hey, I'm done with everything up to this point go ahead and free it Well, the problem could be that if If we need to retransmit then the very next piece The the last part of the TLS record in order in order for the Nick to be able to retransmit that it's got to see the front part So if we didn't have these m buffs We'd have to have we'd have to have come up with some more expensive reference counting way to prevent the front of the TLS record being freed so that the Nick could again DMA it down and And and recalculate to check some for the first part of the TLS record so with all that said the first Basically with for our software TLS implementation We pass the data from from user space into the kernel or from send file into you know into the kernel and the kernel does the TLS framing in the kernel and Like I was hinting at before The m buffs are marked not ready while they're waiting to be encrypted and the Basically the m buffs are our cued onto into basically a per a per CPU TLS kernel TLS worker thread and once that once that worker thread Encrypts the data it marks it ready and it's it's ready to go to TCP Yes, so I'm going to talk a little bit about something called m buff sand tags and This is basically a pointer so when you're doing Nick TLS offload you're Allocating a resource on the Nick to hold the crypto key and the crypto cursor and The sand tag is kind of owned by the Nick and It allows the network interface to decide if the packet coming in needs the special processing or not and The reason we put this sand tag in the m buff is That it needs to be very fast We cannot do a look up in a hash table Mess around with five tuples down in the Nick driver. It needs to be fast so and also We need to be able to traverse technologies like Vlon and lag that's short for link aggregation and You can imagine when a packet goes out That it might not always go out on the same Nick If you reconfigure your lag for example the packet can suddenly change to another interface and It's very unfortunate if suddenly your unencrypted traffic go straight on the wire so We added mechanisms that will detect route changes in both Vlon and log and It will also check if the underlying network device supports Nick TLS offload to prevent unencrypted data going on the wire The API for sand tags is very simple. We basically have four methods And these are function pointers in the network or struct if net and free BSD You can allocate a sand tag you can Modify it you can query it you can free it The allocate function is recursive. So you basically ask your route interface. I Want to have a sand tag for TLS then it checks the capabilities of the network interface Do I have TLS support or not if I don't have TLS support? We return a failure And this is recursive. So so if you have a Vlon on top, that's the Vlon first and for log It's so that log use something called a hash of the five tuple. It's usually called a flow ID and This information is not always present at the beginning of the connection. So so sometimes before you can allocate a crypto tag you need to wait for a Few packets to be exchanged so that the socket can record Which is my hash and which is my then? output network interface under lag and so Again, which is my destination Output ring in the network interface that this is usually called a topletes hash and we used seven least significant bits to switch the packets on the TX rings But maybe a lot of you are familiar with that From the network Stack perspective Things are very simple You basically set the sand tag pointer Which is the sand tag in the packet header of the ambuff and then you also need to set a checksum flag for the sand tag because We tried to avoid increasing the size of the ambuff so it would use another cache line and Unfortunately, we had to share the sand tag with the receive interface pointer and to avoid Leaking receive interface into sand tags for example when you do a ping you might get back to receive interface pointer So it's in the union This is maybe too much details for you But we have a flag we abuse a checksum flag to indicate if you have a sand tag or not From the network driver perspective it basically does the opposite it checks if the checksum flag is set and It does a container of the sand tag you specified in the ambuff and This usually gets you the per network interface specific Structure that contains for example the destination thank you or a copy of the so-called flow ID And it can do a simple check. Is this packet still valid for this interface or not? And this is in the fast path, so here you can see a set of Different use cases for data flow so The the the good old case is that you're using a socket right That is all to the left with open SSL. You have an unencrypted buffer Inside open SSL You do the encryption in user space You copy the encrypted buffer into the kernel and Then again the nick will read from the kernel buffer and put it on the wire In the second case where you have software kernel TLS You have the unencrypted buffer in user space. You write it into the kernel via a system call and The kernel will then encrypted like Drew said earlier We have a per CPU thread that will read unencrypted data with am not ready from the socket buffer and it will encrypt it and put the ready flag and then the encrypted buffer will go on On to the nick and the nick will put it on the wire and Then with Nick kernel TLS you as you can see you have an unencrypted buffer in user space you write it into the socket buffer and After it goes into the socket buffer we write it straight to the nick and I can also mention that there is some magic here Going on. So so when you do a system call write to the socket buffer for every system call you do you will add an TCP no TLS header and trailer. So so it will kind of wrap your transmitted data with Automatically with the TLS Header and trailer. So so it's all seamless to user space. You just write the unencrypted data and It's it's automatically encapsulated and Then after the data is in the kernel with this additional header and trailer Then it will go to the nick and the nick will do the encryption so this this eye chart here basically shows the the data flow for send file and In particular, I'm showing the data flow for send file with software kernel TLS And one thing you'll notice is that for basically every hundred gigabits is you know divide by 8 for gigabytes It's 12 and a half gigabytes a second. So when you're doing When you're doing send file with software kernel TLS basically the data flow is you bring things in from that You bring things in from the disks Into memory and then the CPU has to read everything you just brought into the disk to do the crypto And then once it does the crypto, it's got to write it back out into the memory And then from the memory it's got to write it Or really DMA read it into the nick. So basically you multiply your your band if you want to do 100 gigabits You've got to have 400 gigabits of memory bandwidth or or 60 gigabytes a second Which is basically just about as much as as a Broadwell Xeon can do the nice thing about Nick TLS is all of a sudden see these green arrows that go up and down now You don't see them anymore because all of a sudden you look you don't have to do this This memory read and memory write anymore and your memory bandwidth requirements are cut almost in half and this is important because Certain CPU vendors like to segment their product lines by memory channels and memory speeds And so you can you can maybe go down a product line if you can do Nick TLS Next slide So here you can kind of see What you would expect if you did TCP dump with a modified IPerf that support TLS offload in the kernel So if you do TCP dump on the IPerf client sending the data You will see here the unencrypted data with TCP dump You see zero one two three four five six seven eight nine zero one two three four five six seven eight nine On the server side you will see the encrypted data. So this is exactly the same packet So this is on the client side and this is on the server side And this is just to show you that this is what you can expect from Nick TLS and also software TLS offload so We hit some issues when trying to implement Nick kernel TLS As I said already the Nick is messing with the TCP data But it already does so with TSO So those familiar with the term TSO large send off load The Nick already update the sequence number when it fragment big chunks of data and now it's also Encrypting TCP data as you go along. So so but who says we have to follow the OSD model for everything Retransmission of TLS packets as true said There is a need for re-sending The beginning of a TLS record if you're doing a retransmit in the middle of a TCP packet And this actually cross TLS record As you might remember I said in the beginning of the talk that we have a 16k maximum length of TLS records and that basically means that if you need to retransmit One byte at the end of a TLS record you need to kind of dump Almost 16k down to the nick before you can retransmit that last byte in order to get the right crypto state so So but for the good case you don't do this. So so you might want to consider Using something like rack or Yeah Try to get the trans retransmission rate as low as possible when using TLS to minimize the PCI bandwidth used Then we have some benchmarks, so so let's see what's first Yeah, Drew. So this is this isn't the benchmark section, but it's not actually a benchmark. This is data from a Netflix circa 2016 hundred gig Server, it's basically a Broadwell base zon with 16 cores 32 threads for really fat fast NVMe drives and one Chelsea ot6 Nick and The case on the left is without kernel TLS and the blue bars are served or the Bandwidth we're serving out of the Nick. So as you can see we're maxing out around 40 gigabits a second and the CPU is is It's an average of 75 because it kind of zooms between a hundred and fifty It's it's a case where it's memory It's basically memory memory bandwidth bound and it's miserable for clients because it speeds up and slows down speeds up And slows down and never really never really finds a sweet spot It's just easier to say 75 and stop there, but you know I ramble The second case is the case that Netflix runs today Basically, we're at we serve at 90 gigabits a second That's our that's our target bandwidth for a hundred for a hundred gigabit cash to to allow for a little bit of extra capacity on the link for other things and We're at about a little less than a little less than 70% CPU with software TLS and the right most is with Nick TLS and as you can see the the CPU is cut almost in half for the for the same bandwidth and Again, this is Nick TLS on a Chelsea o t6. That's not available right now in head It's something that John Baldwin has patches for in GitHub and you can talk to him after the After the presentation if you want more information on that So here we have some benchmarks with Malanox Nick TLS so the the orange Line show here Software kernel TLS. So that's basically using the AS AES Instructions in the kernel to do encryption of the packet It's going on the wire and as you can see as you go up by number of threads. So You usually start Congesting the CPU because doing software encryption is relatively heavy even if it's done in the kernel and The blue line on the bottom is what you would get with plain text. So the CPU usage rises Linearly Because on this machine you have like 28 Course available and You see here the as you go up to 28. It's almost linear and then it rises a little bit and the gray line is showing Nick TLS with not yet on the market Malanox connectax 6 DX and You can see it used a little bit more CPU and that's likely because it's Encapsulating it with smaller TLS record. So it needs to encapsulate every 16 Kilobytes well, if you use plain text, you can do 64 kilobytes at a time with regular TSO and So so And and this is also interesting for those of you that do virtualization That you can imagine you can have a virtualized environment Where you don't have to do encryption in software or at the CPU at all You can have a or a weak arm processor can maybe also be a target for such applications So so you only need to switch the packets around and and then the nick will do everything for you Or almost everything except a TCP stack. We still want to do TCP stack in the kernel and As I already mentioned Malanox Has a web page you can go to Theoretically we can support up to 16 million simultaneous TLS records Now TLS streams at the same time either 25 gig 50 gig 100 gig and now also there will be 200 gig a bit second That's various configurations. You can use this It's you can use 200 gig ports or you can have one 200 gig port so There's different ways you can get 200 gig and Yeah, you maybe don't want to say how you get 200 gig over at Netflix Well, I'll talk about that in the next hour. Yeah so Then we have Little bit about Chelsea O's hardware TLS Of flow. Do you want to say a few words John Baldwin or so? basically the the T6 T6 Nick supports TLS one that one that one and one dot two in the in the fairly unique thing is it can do both CBC and GCM We have John has the tow support for the Sorry Colonel TLS support for for the toe mode of Chelsea O in progress. That's not something we use at Netflix But it's interesting to a lot of people and the really interesting thing is that the open crypto framework CCR Chelsea O driver is already Usable in the tree and you can use it, you know, basically right now with what's in the tree for It's actually one of the only It's basically the only thing we're talking about now for hardware offload that you could actually like, you know Use like at this at this second if you're running head So we reached the end of our talk and I would like to know are there any questions Can you go back to the slide with? un Unallocated ones I'm up so so just because I just understanding problem instead of a chain with a pointers You're using just an array to point everywhere to Okay, okay, so so that's isn't a generally good advice to avoid that kind of cues with pointers Yes, put a race in the modern. Yes, it's something that it's something that Linux does with their with their SKB pages I think that's what they call it. It's something that I wanted to do in free BSD for years. Okay. Thank you. Thank you Morning, you mentioned virtualized environments. How will the Nick crypto offload be available to S running inside the hypervisor machine. Well, that's really a good question. I will just repeat so in virtualized environments with melanox nicks They provide virtualization Inside the Nick so so the Nick has multiple virtual PCI functions and you can give this virtual PCI functions to your virtualized instance and And this way You kind of have all the DMA rings Inside your virtual machine and and the Nick will actually read from this rings directly So so that this is one You know unlike Intel adapters That they don't support this so well So with but with melanox cars you can really do it large scale that that You can split up One network card physical network card into many smaller virtual PCI functions And you can just hand them out to your virtual machines. What's that answer for your question? Thank you. I mean more questions So it's it's pretty rare what's what's in what's in head right now is TLS up to 1.2 I actually have a patch for TLS 1.3. That's that's working and has served a real Netflix customer traffic So and and they've also run TLS 1.3 with Nick TLS So it and I think a new TLS version at this point things are becoming ossified You can tell how ossified they're becoming because 1.3. He's got a masquerade as 1.2 So I think that the I think that's going to slow down a little bit. It's just my personal opinion. I mean But you do you do lose flexibility And that's definitely true, but from our at least from a Netflix perspective Except for people watching on a web client We mostly most of the clients are upgraded a fairly slow at fairly slow pace I mean one of our problems is is I'm still supporting TLS 1.0 because a grandma smart TV She bought in 2010 or whatever right so Okay final question