We’re excited to announce that, as of today, masculine and feminine alternative translations are available for when translating from English to Spanish, French, or Italian. You can try out this new feature in both Bing Search and Bing Translator verticals.

Over the last few years, the field of Machine Translation (MT) has been revolutionized by the advent of transformer models, leading to tremendous improvements in quality. However, models optimized to capture the statistical properties of data collected from the real world inadvertently learn or even amplify social biases found in that data.

Our latest release is a step towards reducing one of these biases, specifically gender bias that is prevalent in MT systems. Bing Translator has always produced a single translation for an input sentence even when the translations could have had other gender variations including feminine and masculine variants. In accordance with the Microsoft responsible AI principles, we want to ensure we provide correct alternative translations and are more inclusive to all genders. As part of this journey our first step is to provide feminine and masculine translation variants.

Gender is expressed differently across different languages. For example, in English, the word lawyer could refer to either a male or female individual, but in Spanish, abogada would refer to a female lawyer, while abogado would refer to a male one. In the absence of information about the gender of a noun like ‘lawyer’ in a source sentence, MT models may resort to selecting an arbitrary gender for the noun in the target language. Often, these arbitrary gender assignments align with stereotypes, perpetuating harmful societal bias (Stanovsky et al., 2019; Ciora et al., 2021) and leading to translations that are not fully accurate.

In the example below, you notice that while translating gender-neutral sentences from English to Spanish, the translated text follows the stereotypical gender role, i.e., lawyer is translated as being male.

As there is no context in the source sentence that implies the gender of the lawyer, producing a translation with the assumption of either a male or female lawyer would both be valid. Now, Bing Translator produces translations with both feminine and masculine forms.

System design

We aimed to design our system to meet the following key criteria for providing gendered alternatives:

1. The feminine and masculine variants should have minimal differences except for those needed to convey gender.
2. We wanted to cover a wide range of sentences where multiple gendered alternatives are possible.
3. We wanted to ensure that the translations preserve the meaning of the original source sentence.

Detecting gender ambiguity

In order to accurately detect gender ambiguity in source text, we utilize a coreference model to analyze inputs containing animate nouns. For instance, if a given input text contains a gender-neutral profession word, we only want provide gendered alternatives for it when its gender can’t be determined by other information in the sentence. For example: On translating an English sentence “The lawyer met her driver at the hotel lobby.” into French we can determine that the lawyer is female, while the gender of the driver is unknown.

Generating alternate translation

When the source sentence is ambiguously gendered, we examine our translation system’s output to decide if an alternative gender interpretation is possible. If so, we proceed to determine the best way to revise the translation. We begin by constructing a set of candidate target translations by rewriting the original translation. We apply linguistic constraints based on dependency relations to ensure consistency in the proposed alternatives and prune the erroneous candidates.

However, in many cases, even after applying our constraints, we are left with multiple candidate rewrites for the gendered alternative translation. To determine the best option, we evaluate each candidate by scoring it with our translation model. By leveraging the fact that a good gender rewrite will also be an accurate translation of the source sentence, we are able to ensure high accuracy in our final output.

Leveraging managed online endpoints in Azure Machine Learning

The gendered alternative feature in Bing is hosted on managed online endpoints in Azure Machine Learning. Managed online endpoints provide a unified interface to invoke and manage model deployments on Microsoft-managed compute in a turnkey manner. They enable us to take advantage of scalable and reliable endpoints without being concerned about infrastructure management. This inference environment also enables the processing of large numbers of requests with low latency. Our ability to create and deploy the gender debias service with the latest frameworks and technologies has been greatly improved through the use of managed inference features in Azure Machine Learning. By leveraging these features, we have been able to maintain low COGS (Cost of Goods Sold) and ensure straightforward security and privacy compliance.

How can you contribute?

To facilitate progress in gender bias reduction in MT, we are releasing a test corpus containing gender-ambiguous translation examples from English into Spanish, French and Italian. Each English source sentence is accompanied by multiple translations, covering each possible gender variation.

Our test set is constructed to be challenging, morphologically rich and linguistically diverse. This corpus has been instrumental in our development process. It was developed with the help of a bilingual linguists with significant translation experience. We are also releasing a technical paper that discusses the test corpus in detail and the methodology and tools for evaluation.

GATE: A challenge set for Gender-Ambiguous Translation Examples – Paper

GATE: A challenge set for Gender-Ambiguous Translation Examples – Test set

Path forward

Through this work we aim to improve the quality of MT output in cases of ambiguous source gender, as well as facilitate the development of better and more inclusive natural language processing (NLP) tools in general. Our initial release focuses on translating from English to Spanish, French, and Italian. Going forward, we plan to expand to new language pairs, as well as cover additional scenarios and types of biases.

Credits:

Ranjita Naik, Spencer Rarrick, Sundar Poudel, Varun Mathur, Jeshwanth Kumar Chandrala, Charan Mohan, Lee Schwartz, Steven Nguyen, Amit Bhagwat, Vishal Chowdhary.

The post Bing’s gendered translations tackle bias in translation appeared first on Microsoft Translator Blog.

Last year Microsoft announced the release of its Neural Network based translation system for 10 languages: Arabic, Chinese, English, French, German, Italian, Japanese, Portuguese, Russian, and Spanish. Today, Korean is being added to the list.

Neural Network translation uses the full context of a sentence to translate words based not only on a few words before and after it, but on the full sentence, generating more fluent and more human sounding translations. This new AI-powered technology delivers the most significant improvement in machine translation quality since statistical machine translation became the industry standard 10 years ago.

Thanks to these improvements in quality and fluency, translations are the closest they have ever been to human generated ones.

HOW IT WORKS

At a high level, Neural Network translation works in two stages:

1. The first stage models the word that needs to be translated based on the context of this word (and its possible translations) within the full sentence, whether the sentence is 5 words or 20 words long.
2. The second stage then translates this word model (not the word itself but the model the neural network has built), within the context of the sentence, into the other language.

Neural Network translation uses models of word translations based on what it knows from both languages about a word and the sentence context to find the most appropriate word as well as the most suitable position for this translated word in the sentence.

One way to think about neural network-based translation is to think of a fluent English and French speaker that would read the word “dog” in a sentence: “The dog is happy”. This would create in his or her brain the image of a dog. This image would be associated with “le chien” in French. The Neural Network would intrinsically know that the word “chien” is masculine in French (“le” not “la”). But, if the sentence were to be “the dog just gave birth to six puppies”, it would picture the same dog with puppies nursing and then automatically use “la chienne” (female form of “le chien”) when translating the sentence.

Here’s an example of the benefits of this new technology used in the following sentence: (one of the randomly proposed on our try and compare site: http://translate.ai)

M277dw에 종이 문서를 올려놓고, 스마트폰으로 스캔 명령을 내린 뒤 해당 파일을 스마트폰에 즉시 저장할 수 있다.

Traditional Statistical Machine Translation would offer this translation:

“M277dw, point to the document, the paper off the file scan command Smartphone smartphones can store immediately.”

Neural Network translation, in comparison, generates this clear and fluent sentence:

“You can place a paper document on M277DW, and then save the file to your smartphone immediately after the scan command.”

The Neural Network translation systems are available for you to use through many entry points:

• Browser: We’d love your feedback on the new Neural Network Korean translation system vs. the legacy statistical one! Visit our try & compare site: http://translate.ai
• Microsoft Translator live feature: When using our new personal universal communicator feature, Microsoft Translator live, neural translations will also be used. For instance, if you use this feature to translate a live presentation from any of the nine supported speech languages to any of the 11 NN-powered translation systems, subtitles will be delivered using Neural Network technology: http://translate.it 
• Instant Messages in Skype: Translate instant messages (from and to any of these 11 languages) using the Skype Translator feature in Skype desktop and Skype Preview for Windows 10.

In addition, developers can easily integrate Microsoft Translator Neural Network systems by using the category “generalnn” in their API calls. There is no extra cost in using our neural network models vs. the existing statistical ones so don’t hesitate to use them!

For speech translation projects, the Microsoft Translator speech API already uses neural network translations from any of our 9 speech translation languages to all the 11 neural network powered languages.

Learn More:

How Neural Network translation works?
Microsoft Translator live feature: the personal universal translator
Microsoft Translator apps
Microsoft AI

The post Korean Becomes Microsoft Translator’s 11th Neural Network Translation Language appeared first on Microsoft Translator Blog.

Using the plugin, visitors can translate a site into any of the 40+ supported languages in one click without leaving the page once this light-weight, cross-browser plugin is installed. This plugin also provides options for a setting a color scheme, as well as an option to allow visitors to suggest translations.

The Bing Translator plugin should be installed from within the WordPress Dashboard by clicking on Plugins >Add New and search for “Bing Translator” and works on any WordPress site. A site developer can also manually install the plugin by downloading it from WordPress.org, then adding the “bing-translator” folder in the “/wp-content/plugins/” directory.

Using Bing Translator Plugin for WordPress Video

More Links to Get Started

• Blog Post from Microsoft Open Technologies
• Translator Plugin Site on WordPress.org
• Installing & Configuring the Bing Translator for WordPress
• Using the Bing Translator for WordPress
• Adding Bing Translator to WordPress using WordPress Widgets
• Adding Bing Translator to WordPress using Shortcodes
• Adding Bing Translator to WordPress using PHP Template Tags

Congratulations to Microsoft Open Technologies, Inc team for their great work on the Bing Translator Plugin for WordPress! 

The post Bing Translator Plugin for WordPress Enables Webmasters and Developers to Localize Site Content appeared first on Microsoft Translator Blog.

Now users can leverage the power of speech-to-speech translations from any Windows device. Simply speak into your device by using the microphone feature to place orders or ask for directions, and hear the translated words in a native speaker’s accent.

In addition to speech input, this new release of the Translator app now offers users the option to use the camera feature in both portrait and landscape mode. Simply point your camera, scan and translate printed language using your tablet or PC to create subtitles for everyday life. 

 
Today, we are also releasing new updates to the Bing Translator app for Windows Phone 8 which include improvements to the speech functionality for better quality and responsiveness of translations, in addition a redesigned user interface for the existing and recently released offline language packs. By downloading offline language packs, you can maintain translation on the go when not connected to the internet and avoid expensive data roaming charges.

You can now download the free app for Windows from the Windows store here and for the Windows Phone from the Windows Phone store here.  Existing users who have already downloaded the app, will be able to access the new updates without needing to download it again. Whether on your Windows Phone or any Windows device, the Translator app is the perfect travel companion to help overcome language barriers, even when there’s no internet connection. To learn more about Bing Translator apps, check out the Translator for Windows and the Translator for Windows Phone product pages.

These apps will become your window to the world, no matter where you are.

The post Say ‘Hello World!’ Announcing Speech Input for the Bing Translator app for Windows appeared first on Microsoft Translator Blog.

The song translation app allows users to upload their favorite songs with lyrics, add timestamps to the lyrics, then translate and record the song in another language. Song Translator leverages the Translator Control while also leveraging other key Microsoft technologies and features including: Windows 8, Visual Studio, Background Task for live tiles, and Windows Azure Cloud Storage, as well as pitch synthesis from SonicAPI.

Built in Visual Studio 2012 with C# and XAML using the Windows 8 Store App template, the app calls the Translator API to process the translations into over 40 of the Translator supported languages.

Watch the Song Translator Demo from Michelle, Priya, and Kat

 [View:http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-91-16/3364.Song-Translator-Demo.mp4]

 How to Use the Song Translator

1. Download the app
2. Unzip the project
3. Install the Nuget package for the Mobile Service
4. Open the configurations file and input credentials

Check out the walk through guides written by our Interns to learn more.

• Using the Song Translator App
• Leveraging Azure Storage
• Using Background Tasks for Live Tiles
• Song Synthesis and SonicaAPI

To learn more about our talented group of summer interns and their experience, make sure to check out the Microsoft Research blog post.

.

The post Summer Interns Singing a Happy Tune with Song Translator App appeared first on Microsoft Translator Blog.

Thousands of developers are at BUILD 2013 in San Francisco this week where Microsoft is showcasing how they can create great experiences for their consumers on Windows platforms by utilizing these technologies in their own applications.

Today during Steven Guggenheimer’s keynote at BUILD, Microsoft showcased the availability of an exciting new update to Twitter for Windows Phone – bringing instant translation of Tweets that are in a different language than your own. Over the last year, Microsoft has been working with the team at Twitter to explore how its translation technology, based on Microsoft Research’s extensive advancements in machine learning, can help the global Twitter community better communicate across language barriers.

With this update, a soccer/football fan can still follow the news about their favorite soccer team even if the breaking news on Twitter is not in their language. Tapping on a Tweet with a globe icon, which indicates translation is available, expands the Tweet and shows translated text right below the original content. The built-in Tweet translation feature is available for the 38 languages supported by the app powered by Microsoft Translator. Download/update your Windows Phone Twitter app to try it out for yourself!

“Breaking down language barriers with world-class research and engineering has been the guiding principle behind the development of Microsoft Translator, and Twitter is an excellent new addition to community of customers and developers leveraging Microsoft’s translation technology for their users,” said Peter Lee, Corporate Vice President of Microsoft Research US. “The integration of machine translation technology from Microsoft Research has the ability to broaden any application’s impact through a substantial increase in accessibility to real time communications and information sharing. No longer is language a barrier to real time instant connections around the world.”

Windows Phone application developers can take advantage of the Microsoft Translator API to bring the power of instant translation to their apps. Windows developers can also download the just announced Translator control for Windows to reach a global audience and differentiate their Windows applications.

As the next billion users come online, we look forward to delivering and enabling many more global experiences by continuing to harness the innovations coming out of our research work and data platforms with developers, app builders and partners. 

The post From Tweet to Translate: Microsoft’s translation service powers new translation feature in Twitter for Windows Phone appeared first on Microsoft Translator Blog.

Today at BUILD, Microsoft announced the preview of the Bing Translator Control for Windows 8 and Windows 8.1.

With the Translator control developers get easy access to robust, cloud-based, automatic translation between more than 40 languages. The Translator control gives the Windows 8 developer access to machine translation services, which is built on over a decade of natural language research from Microsoft Research. After download and one-time authentication, you can simply place the Bing Translator control in your app, feed it a string to translate, and receive the translation. 

The Bing Translator control, powered by the flexible Microsoft Translator API puts the power of a world class machine translation system in your hands. Designed to enable a variety of scenarios, it is available in a number of developer friendly protocols.

The control takes the text you pass in to the Bing Translator web service for translation, and then passes the translation of the input back to you. Your application can handle the return text however you see fit. You receive the text to translate by any method, and render it by any means you choose-  using the Translator Control in the middle, in order to translate the input text to the target language.

Get started with the Translator Control today!

Download the Translator Control

Sign Up for the Translator Service

Get Samples and Documentation

The post Announcing Today at BUILD – Bing Translator Control for Windows 8 and Windows 8.1 appeared first on Microsoft Translator Blog.

You can now download the free app from the Windows store here.

The Bing Translator app is based on years of Microsoft Research’s investments in advancing machine learning – a way to find patterns that humans can’t see, helping people interpret the words and worlds around them.

Translating content whether browsed, typed or scanned is nearly instantaneous. Just point your device’s camera at printed text and watch as the translation is automatically overlaid over the video stream – creating subtitles for everyday life. You can also type to translate with your keyboard and hear translations spoken with a native speaker’s accent.

The Translator app is the perfect companion when traveling. The app can help overcome language barriers, even when there’s no internet connection. Save on expensive data plans when traveling with offline language packs for select languages so you can travel with confidence, even in the most remote locations. More language packs coming soon.

The Share Charm lets you quickly translate highlighted text in any Windows 8 app, with Snap View you can multi-task while browsing, chatting or more by snapping Bing Translator to the right or left of your screen. With this unique feature, powerful translation technology is just a swipe away in Windows 8 no matter where you are – at your desk or on the go.

For more about the app, check out the Translator for Windows product page, and check back for future developments and updates.

We hope that this app becomes your window to the world, no matter where you are!

– Vikram Dendi,
Director of Product Management,
Microsoft/Bing Translator – Microsoft Research

The post A Window to the World, Bing Translator App for Windows Now Available appeared first on Microsoft Translator Blog.

Klingon^* is now a supported option on the Bing Translator site, allowing you to translate text snippets and web pages to and from Klingon.  Bing Translator for Windows Phone added Klingon as a supported language, for text mode input/output and camera mode output. On the Bing Translator site, you can also choose to translate to both Latin-script Klingon and to plqaD (the Klingon script). Please note that if you are translating from Klingon, you would need to explicitly select the language (rather than rely on Auto-detect).

This system has been built as a labor of love, in close partnership with members of the Klingon Language Institute (KLI) headed by Dr.Lawrence Schoen, Prof. Marc Okrand, the inventor of the Klingon language, many other Klingon enthusiasts inside and outside Microsoft. We received fantastic support from our fellow Star Trek fans at Paramount and CBS.

Building a new translation system from scratch is a challenging affair, requiring a large amount of training documents, many iterations of training the engine, reviewing and evaluating, and repeating this many times. What you initially get is mostly unintelligible, and with continued learning comes the improvement – both in vocabulary and in fluency. While there is a great amount of training material for such a system in mainstream languages like English, French or German, Klingon is a language that does not (yet!) have a comparable volume of “parallel” (translated) text, or even material in Klingon alone. Our friends in the community were able to help us gather what is available, and used the Microsoft Translator Hub to train the initial engine. Members of the community were then able to review, critique and correct the translation errors this infant system was making. These corrections directly influenced the next training run, and thus the system has been getting better every day. Given its infancy, and the distance it has yet to travel to achieve the necessary fluency and vocabulary – Klingon will stay as an experimental language in Bing Translator for the time being.

We wish to thank the Klingon language community, Prof.Okrand, Dr.Schoen and CBS/Paramount for helping make this a reality. If you are a Klingon speaker and wish to join the Hub community built around this effort, please email lawrence@kli.org or translator@microsoft.com. Not everyone can have Lieutenant Uhura translate for them, so we hope Bing Translator’s Klingon support comes handy next time you are in a pinch.

lupDujHomwIj lubuy’moH gharghmey

– Vikram Dendi & the Translator team at Microsoft

Update (2:52 PM): Added note about auto-detection, and other minor edits.

* Klingon is a trademark of CBS Studios Inc.

Here is a translated version of the original Klingon Empire Announcement:  

SIbI’ maqlu’

Immediately proclaimed  

DaH tlhIngan Hol mughlaH Bing Translator ‘e’ maq tlhIngan yejquv

The Klingon High Council announces that Bing Translator can now translate Klingon

tlhIngan Hol ‘oH qIb Hol wa’DIch’e’ mughlaHbogh Bing Translator ‘e’ maq tlhIngan yejquv, boqbogh tlhIngan Hol yejHaD, Microsoft je.

 Klingon is the first galactic language which can be translated by Bing Translator, announces the Klingon High Council, in alliance with the Klingon Language Institute and Microsoft.

qaStaHvIS DISmey, yuQjIjDIvI’ luSuchtaHvIS tlhInganpu”e’, Qatlhqu’ tlhIngan Hol mughmeH ‘ej tera’ Holmey mu’tlhegh lIngmeH Qu’, nuja’ tlhInganpu’. tera’ Holmey rurbe’chu’ tlhIngan Hol, ‘ej ‘oH HaDtaH tera’ngan law’. wejmaH tera’ Sep, Hoch puH’a’ je Dab HaDwI’pu’. qIb ghatlh tlhIngan Hol, tlhIngan tIgh je ‘e’ ‘agh ngoDvam.

For years, Klingons have told us that the task of translating Klingon and producing sentences in Earth languages while visiting the UFP is very difficult. Klingon is truly unlike Earth languages, and many Earthlings (continue to) study it. Students (of Klingon) live in thirty different Earth regions (countries) and all great landmasses. This fact demonstrates the galactic dominance of Klingon language and the Klingon Way.

tlhIngan Hol chelta’mo’ Bing Translator, qIb lengwI’vaD, tlhIngan wo’ SuchwI’vaD je nuH ‘ut mojbej mughwI’. Hoch SepDaq, tera’nganvaD tlhIngan Hol, tlhIngan tIgh je lIH Bing Translator mughmeH laHmey. pIj mughwI’ lo’chugh taghwI’, nom tlhIngan Hol pab pIn moj.

Because Bing Translator has added Klingon, the translator will certainly become and essential weapon (tool) for (the) galactic traveler and (the) visitor to the Klingon Empire. In every region (country), the translation abilities of Bing Translator will introduce Earthlings to the Klingon language and the Klingon Way (culture). If beginners frequently use the translator, they will quickly become grammarians of the Klingon language.

Qo’noS Qombogh muD, tuj’a’, Debmey tIn je SIQlaH tera’nganpu’. pIraQSIS Qaw’lu’mo’ choHpu’ Qo’noS ‘e’ leghlaH je. Bing Translator lo’taHvIS lengwI’, lengDI’ bel, ‘ej roD batlhHa’ vangbe’laH ‘ej tIgh chach junlaH. Microsoft Bing Translator, qum chaw’ je ghajchugh «SuvwI’ lengmey» lengwI’, tlhIngan SuvmeH tIgh ‘ut ghojlaH, qagh SoplaH ghopDu’Daj lo’taHvIS, ‘ej pIjHa’ QumHa’.

 Earthlings will be able to endure (experience) the quaking (turbulent) atmosphere, great heat and large deserts of Qo’noS. They will also be able to see that Qo’noS has changed due to the destruction of Praxis. While the traveler uses Bing Translator, he will be comfortable while travelling, and will usually be able to not act dishonorably and avoid cultural emergencies. With Microsoft Bing Translator and a government permit, “Warrior Tours” travelers can learn essential Klingon fighting, eat qagh with their hands and infrequently miscommunicate.

che’ronDaq mughwI’ mu’tlheghmey, mu’mey je tobta’ tlhIngan Hol yejHaD. jIjDI’ tlhIngan Hubbeq, ‘ejyo’ je, toy’beH mughwI’. ‘e’ poQbej SuvwI’ Hol. DaH not Hegh SuvwI’ «HIjol» mughHa’DI’ boq beq ‘ej «HIQoj» mojDI’. taHmeH tlhIngan wo”a’ HoSghaj, lI’chu’ Bing Translator mughmeH laHmey.

The Klingon Language Institute has tested the translator’s sentences and words on the battlefield. When the Klingon Defense Force and Starfleet cooperate, the translator will be ready to serve. A warrior language certainly requires that. Now warriors will never die when “Beam me up!” is mistranslated by an alliance crew and becomes “Beam me out!” In order that the powerful great Klingon Empire continue, the translation abilities of Bing Translator will be supremely useful.

pItlh.

The post Announcing Klingon for Bing Translator appeared first on Microsoft Translator Blog.

With the proliferation of digital content on the web, mobile devices, and desktop applications, there is an increasing demand to communicate and collaborate in multiple languages. Helping enable business, communities, and consumers to communicate and collaborate across language barriers through technology innovation is a core focus for the Microsoft Translator team.

Today, I am pleased to announce the launch of two new officially supported languages: Malay and Urdu. These two languages join the other languages already supported by the Microsoft Translator platform and Bing Translator. Malay is spoken by over 200M people worldwide in countries ranging from Malaysia to Brunei. Urdu is spoken by over 100M people worldwide and is spoken by large populations residing in the Indian subcontinent, the Middle East and countries in Europe and North America. It is the national language of Pakistan and the official language of several states in India.

A year ago, on the last International Mother Language Day, we announced the release of Hmong as part of a close engagement between Microsoft and the Hmong community – a small but significant step towards empowering businesses and organizations to tap into the power of Microsoft’s language technology. Like Hmong, the development of Urdu is the result of a community effort shepherded by the Jawaharlal Nehru University (New Delhi, India) under the leadership of Dr.Girish Nath Jha, and Microsoft, utilizing the powerful Microsoft Translator Hub customization tools.

In addition to the launch of these new languages; we are also rolling out several new improvements to our platform, customization tools, and language quality. See the release notes for this release in our forum here.

We have seen some great momentum with both the business and language communities for the Translator Hub. Through the Hub, users are able to bring better and specialized translation quality to established languages, as well as the many native languages of the world that are not yet supported by major translation providers which go to the core of supporting the goals of Mother Language Day. Urdu is the latest language community benefiting from the availability of the Hub.  If you are passionate about the community development efforts around Urdu or other languages that we support and want to become involved in the efforts, please contact us.

Commemorating the International Mother Language Day, Microsoft Local Language Program (LLP), also announced the support of 13 extra languages to our range of Language Interface Packs (LIPs), bringing the total number of languages supported by Windows 8 and Office to 108. Learn more at the LLP website.

– Vikram Dendi,
Director of Product Management,
Microsoft/Bing Translator

The post Celebrating International Mother Language Day with the Launch of New Languages & Features appeared first on Microsoft Translator Blog.