Tag: openUTAU

Setting Up Multipitch, Multiexpression Voicebanks in OpenUTAU

Introduction to Multipitch, Multiexpression Voicebanks in OpenUTAU (as of January 2022)

Before We Begin

This tutorial is a bit much, and it acts as Part One to our Voice Colors blog! Today, we want to go over how to set up Multipitch, Multiexpression voicebanks in OpenUTAU. The tutorial assumes you understand the following:

  • How Multipitch voicebanks work in UTAU
  • How to use and set up a Prefix.map
  • Suffixes and Prefixes, and how to use them to create Multipitch

This tutorial also assumes:

  • You have operational voicebanks prepared
  • All of your pitches and expressions, for each voicebank, have a different suffix

Lastly, we are not experts! We are learning along with our community. There may be more than one way to perform the following steps, and things may change in the future as OpenUTAU develops. We will do our best to keep this blog as up-to-date as we can, and we will come back to make edits when necessary. Knowing that, let’s dive in!

Getting Started

We know now that it’s possible to set up Multipitch, Multiexpression Voicebanks in OpenUTAU. Today, let’s go a step further and use our knowledge about Voice Colors and PrefixMaps. Let’s get into the nitty-gritty and set up something a little more complex! 

For this example, I will be porting over an existing UTAU with five different appends/expressions. As this is not a STUDIO OGIEN created UTAU, we will be redacting the name to avoid confusion. We do not manage or act as user support for this voice, and the following tutorial is only for educational purposes. Now, let’s set this up!

To start, I have downloaded all 5 expressions as well as organized and extracted them on my Desktop. For the sake of sanity, I have also prepared six new folders for extra samples to help keep organized during the process. Make sure to keep the original compressed files as a backup, just in case.

Multipitch, Multiexpression Voicebanks in Open

Let’s begin inside Voice 1’s folder. I have Voice 1_extras open alongside it. Let’s delete the $read. Next, if you have breaths and other samples, let’s sort them into the _extras folder for now. 

Alright, now that we have cleaned up, we are left with the following three folders, each containing a unique pitch.

Multipitch, Multiexpression Voicebanks in Open

For the sake of organization, let’s rename these folders. For this example, I will name this Voice 1_[original folder name]. Go ahead and repeat these steps with the next expressions.

Multipitch, Multiexpression Voicebanks in Open

Finding Pitches For Mystery Files And Unsorted Folders

As I began to clean up Voice 2’s folder, I noticed a small situation. We have a bit of a dilemma on our hands!

Multipitch, Multiexpression Voicebanks in Open

One of the pitches for Voice 2 is in the main directory, it hasn’t been labeled with the pitch’s name, and there is no Suffix in the oto.ini. If this is your own UTAU, and you know what the pitch is, go ahead and throw these samples into their own labeled folder. If this is not your UTAU, make sure you check the readme files provided to figure out the pitch. However, let’s say there was no documentation to help. We can still find the pitch, even if we’re not musically inclined. I will show you how to find the pitch using REAPER!

Multipitch, Multiexpression Voicebanks in Open

After opening, drag and drop a sample into the track. To understand what pitch this sample was recorded on, I’m going to add an FX to the track called ReaTune.

Multipitch, Multiexpression Voicebanks in Open
Multipitch, Multiexpression Voicebanks in Open

Hit OK, and you will see a new window pop up. Play the track and watch REAPER works its magic.

Multipitch, Multiexpression Voicebanks in Open

The pitch will waver, so pay attention and try to find the average. If you can also find documentation on the voicebank, that will help a lot as well. Create a new folder with the pitch’s name, and move the voice files into it. Let’s proceed!

What To Do With Unique Samples and Other Files

What do we do with our extras, now that our folders are set up? Well, that depends, and it can be done in many different ways. For this particular Multiexpression OpenUTAU voice, I’m going to put in a little more manual effort to make sure these notes always work.

Let’s open Voice 1_extras back up alongside Voice 1. 

Multipitch, Multiexpression Voicebanks in Open

We want to still save the readme files, the character file, and the icon to merge together later on. For now, I will leave those in this folder. However, for the voice samples, it’s a little more complicated. In this example, the extras are recorded for only one pitch. Normally, these would be useable in regular UTAU, but with the nature of these Multiexpression OpenUTAU configurations, we need to define them more clearly for the program to understand what’s going on. So, gather your Suffixes and get ready to copy-paste, and copy-paste a lot at that.

Multipitch, Multiexpression Voicebanks in Open

Inside the oto.ini, we can see the extras lack a Suffix. That’s fine, as one pitch in this particular expression does as well. For that, we will copy paste the voice files into said folder. Once copied over, open the oto.ini file within that folder, and copy+paste the lines for your extras within. Now, this pitch knows how to use these samples.

For the next pitch, you will have to add the suffix in by hand. Once you’ve added in the suffix, copy and paste those oto lines into the oto for the next pitch. Example:

Multipitch, Multiexpression Voicebanks in Open

Copy and paste the extras into the folder. For the next pitch, just hit Ctrl H to bring up Notepad’s search and replace. Just be careful what you put into it. Always double check your work!

Multipitch, Multiexpression Voicebanks in Open

Configure the rest of the extras and we can proceed!

Putting It Together With Voice Colors

Now that our files are ready, let’s move them into a brand new folder. Create a new folder, this one you can name whatever you please. For this example, I will call it “VOICE_Master”. VOICE_Master will need a few special files.

First, let’s gather all the readmes, character files and icons. Decide what icon you would like to use, and set up the character.txt accordingly. Update the name, author, image, web, etc. as you see fit. Next, it would be best to label each readme to refer to the proper voicebank, or combine them all into one. Once finished, you will want to place these within the VOICE_Master folder.

Next, it’s time to place each pitch into the main directory. PLEASE NOTE: You must place individual pitches into the main directory. Splitting the pitches up by expression (ex: VOICE_Master/Voice 1/1_G3) will not work, and OpenUTAU will split the expressions up. Rather, you want your setup to mimic the one below:

Multipitch, Multiexpression Voicebanks in Open

Let’s compress this bad boy and import it into OpenUTAU!

Multipitch, Multiexpression Voicebanks in Open
It works!

SUCCESS! The files are recognized as one voicebank. Exactly what we need for part two.

From here, you can follow our tutorial on OpenUTAU voice colors to achieve the desired effect.

Setting Up Voice Colors in OpenUTAU

About OpenUTAU’s Voice Colors

Voice Colors (also referred to as subbanks) are an exciting new direction for the OpenUTAU software, improving upon the usage and development of multi-expression voicebanks. Until now, users of UTAU have had a strict way to go about using Appends. For a monopitch voicebank with the additional expressions added in, creators of UTAU voices could configure appends with suffixes. Users could then plug the suffixes into the SuffixBroker within the software, or manually add the suffix to each note to achieve the desired effect. For example, a “Soft” append may include the suffix “S”, users would then add this letter onto the end of a note to make the software play that notes for that specific append.

The Old Way

To achieve a multi-pitch, multi-expression voice, creators would have to either split up each append (ex: soft, power, and the base voice all released and used separately) OR the creator of the voicebank would have to use the same pitches for each expression to be able to use all three voicebanks at the same time (ex: all pitches of soft, power and the base voice would all be recorded at A3, C4, C5). Such an example is our very own KASAI OG01 SALIENT voicebank. 

These issues add difficulty when developing new voicebanks, as many voice providers find it difficult to keep the same range for such different tones of voice. Luckily, we no longer have either of those problems with OpenUTAU Voice Colors. Essentially, every Color can have its own, unique prefix-map. In short, this gives users the ability to assign different ranges and suffixes to different voicebanks. 

An Example

Let’s say a voicebank includes three appends; Soft, Power, and Base. Let’s also assume each of these voicebanks has three pitches, but they all use a different set. Soft might use F3, A3, C4. The base may use A3, D4, and F4. Meanwhile, Power might include G3, E4, and A4. That’s very complicated and pretty much impossible to use together in UTAU. Some samples would be used on totally different ranges than they were meant to be used for, causing a rather chaotic sound. In this scenario, within UTAU, it’s better just to split these voices up to use individually.

HOWEVER, with OpenUTAU’s Voice Colors, we can now assign all three appends to their correct ranges. Soft can use F3, A3, and C4 where they are meant to be used, and the same goes with the other two.

OpenUTAU Voice Colors Tutorial
An Example of Defined Voice Colors

The New Way

In comes OpenUTAU’s Voice Color feature! A handy new tool that gives UTAU creators a greater scope of diversification for their expressions and range. The Voice Color feature is essentially the SuffixBroker’s natural evolution, becoming more and more like commercial vocal synthesis products. Users no longer have to plug Suffixes in individually or through a series of menus. Rather, simply make sure “CLR” is available to use at the bottom of a track, and make sure you are using a UTAU configured for Voice Colors. If not, we’ll teach you how to set that up in just a minute.

OpenUTAU Voice Colors Tutorial
Voice Colors in the CLR Editor

Okay, so what makes this so great? Just how easy is it to use? Well, that’s a simple question with a simple answer. By just the press of a button, or a click and drag, users can quickly and easily alter numerous suffixes at the same time. No more rooting around in readme files to copy and paste special characters!

OpenUTAU Voice Colors Tutorial
Example of Working Voice Colors

Awesome, how fun! Now we have greater flexibility than ever before to develop interesting and unique voicebanks. However, as OpenUTAU is still very fresh and new, it is going to have some issues as it continues to evolve. One such issue is setting up Voice Colors themselves. There’s not a lot of documentation or help just yet on that, so setting OpenUTAU’s Voice Colors up? A little tricky.

Setting Up Voice Colors in OpenUTAU

Let’s go through the process of setting up Voice Colors! The developers behind OpenUTAU have made this process pretty darn easy. Today, I am working on porting over APOLLO OG0X, so she will be the voicebank used in this example. Let’s go to Tools > Singers.

OpenUTAU Voice Colors Tutorial
Tools > Singers
OpenUTAU Voice Colors Tutorial
The Singer Window

Ah, here we are in the lovely Singers window. Here, we can see a list of all our subbanks, their aliases, Sets (folders), sample names, phonetics, and prefixes. Extend the window out to see even more information!

OpenUTAU Voice Colors Tutorial
The Singer Window, Expanded

Let’s set up some Voice Colors. Go ahead and click “Edit Subbanks”.

OpenUTAU Voice Colors Tutorial
The Edit Subbanks Window

Alright, here’s where the magic happens, and things get a little tricky. APOLLO OG0X’s VCV includes three voicebanks. Original, Breaking, and Murmur. Let’s set those up, so I can show you an issue you may stumble upon. Click “Add Color” and enter the name you wish. I’ll start with “Original”, her default singing voice.

OpenUTAU Voice Colors Tutorial
Naming a Color
OpenUTAU Voice Colors Tutorial

This one is a super easy monopitch voicebank, as it has no suffixes or prefixes. Simply hit save. 

OpenUTAU Voice Colors Tutorial
A Voice Color, Saved

Boom. Done. One color down. Now, let’s set up an append. Click “Add Color”, and name the append like before. Now, grab the suffix used in the oto.ini for this specific voicebank. Hit “Select All”, add the Suffix into the Suffix box, and hit “Set”.

OpenUTAU Voice Colors Tutorial
Select All, Enter Suffix, Click Set…

Looks good. Hit save and repeat for all appends.

OpenUTAU Voice Colors Tutorial
Defined Voice Colors

Voice Color Issues

All right, everything looks good, now let’s test it!

OpenUTAU Voice Colors Tutorial
An Error. The Suffix For “Breaking” Will Note Populate.

Ah… Uhm… Remember that tricky bit I mentioned beforehand? Yeah, this is it. As OpenUTAU is still in the early stages of development, it’s going to have some little bugs here and there. Notice how “Breaking”, APOLLO OG0X’s power bank, doesn’t assign her suffix to the note? At this time, on January 18th, 2022, OpenUTAU prioritizes Voice Colors alphanumerically. If the name of the default voicebank comes after the name of an append in the English alphabet, OpenUTAU assumes it is the default voice and won’t add a Suffix. This is just a simple override issue that may change! 

For now, let me show you a quick, easy fix. Go back through Singers > Select the UTAU > Edit Subbanks. For this instance, I’m selecting Original.

OpenUTAU Voice Colors Tutorial
Renaming A Voice Color

Simply hit “Rename”, and give it a name that, alphabetically, comes before the names of any append. For this example, I chose “Base”. I’ve also seen others simply use quotes (“”), which also works. Click “OK” and then “Save”. Go back to your track to check….

OpenUTAU Voice Colors Tutorial
Woohoo! Working Voice Colors!

There we go, all fixed! So long as the default voice has a name that comes first alphanumerically, it will stick to the bottom of the Voice Colors list and should function properly. As OpenUTAU develops, this trick may become unnecessary, but, for now, it is here to help, and so are we!

A Quick Tip

You set up your voice colors, you tested them and they worked, but when you loaded a UST… What the heck!? Why aren’t they working!?

Ugh, Not Again!

Here’s a nice quick fix. You may notice the UST imported in this example is in VCV format. Hit Ctrl + A to select all. Go to Lyrics > Japanese VCV to CV. This will turn the UST back to CV, and OpenUTAU will automatically convert the notes into VCV as seen below. We have only tested this with VCV voicebanks at the moment, so if you are employing a different method, you may need to experiment!

It’s Working Again!

LET’S HAVE FUN WITH VOICE COLORS!

Check back soon for more tutorials on OpenUTAU!

Need more assistance with UTAU and creating your very own voicebank? STUDIO OGIEN has compiled resources to use with the UTAU software. Check it out here! If you can’t find what you’re looking for, please let us know through our contact form or leave a comment on this article. We can’t wait to see what you create!

What is OpenUTAU?

About OpenUTAU

OpenUTAU is an open-source project created by StAkira, meant to improve upon the original editing environment for the UTAU software. The software provides a modern user experience, building upon the initial concepts originally introduced in UTAU. As an open-source program, it is available on GitHub. Contributors can assist in the development of this exciting new engine. Many other developers have contributed parts as well at this time, aiming to create a true community UTAU tool!

Editor
Preview of OpenUTAU from the project’s page

What OpenUTAU Is Striving To Be

Above all, the main focus of the OpenUTAU project is to create a modern UTAU user experience without replicating exact UTAU features. Certainly, it is planned to boast an easy-to-use plugin system, a smooth preview/rendering experience, an efficient sample connecting engine (wavetool), and an improved resampling engine interface.

OpenUTAU aims to grant a few of the UTAU community’s long-held wishes as well, bringing it up to speed with more advanced programs. It will feature select compatibility with UTAU technologies, including intelligent VCV and CVVC. The program will automatically convert CV to VCV. If a VCV sample isn’t available in the voicebank, it will fall default to CV. 

Additionally, perhaps the most exciting feature, Internationalization, will include UI translation and file system encoding support! What does this mean? In short, you should prepare for an easier user experience. Complex languages in UTAU, such as Arpasing English, will be easier to use than ever. Forget [l ih v], just enter “live”! The software does the rest. At this time, select languages and voicebank styles are compatible with this technology.

Currently, OpenUTAU presents a rather transparent operation that aims to keep the user in the loop. In addition to a Discord server for users and developers, the engine notifies the user with update notifications. These updates install directly from within OpenUTAU. All in all, the software appears to be on its way to a more powerful, community-based tool that UTAU fans have been wishing for. We are excited to see where it goes!

OpenUTAU automatically checks for updates on launch!

What It Is NOT

At this time, it appears the scope of OpenUTAU does not include its own resampling engines (a.k.a resampler), a full-featured digital music workstation (ex: mixing and mastering), nor does OpenUtau intend to be a Vocaloid duplicate (save a few similar features).

You Could Help Shape The Future Of OpenUTAU!

As open-source software, OpenUTAU was literally made to be a community endeavor! Developers across the globe are able to contribute and improve upon the project. Currently, users can report issues through Discord or Github directly to the creators. For those with skills in coding, contribute fixes through pull requests! The team has also set up a Trello board to see the engine’s progress.

Development Resources:
Discord | GitHub | Trello

Voicebank Progress


Honos VALOR
12%
Apollo PRIME
0%
Theia MONARCH
0%
-->