Can AI generate music that’s worth listening to? Or does AI-generated music suck?
Music is one of our oldest art forms, and art is creative. We tend to think of people as creative, but what about computers … or AI?
It might surprise you, but I’ve been listening to AI-generated music for much of the past few days. And while I wouldn’t listen to it all the time, it’s been great to work to and enjoy.
In this episode of The AI Show with John Koetsier, we chat with Edward Balassanian, CEO of Aimi.fm. We chat about what the Aimi.fm team has built, how they did it, what AI technologies they used, and where he plans to take the company next.
Listen: AI-generated music
Don’t forget to subscribe on your favorite podcasting platform:
Watch: AI-generated music
And … full transcript: AI-generated music
John Koetsier: Can AI generate music that’s worth listening to?
Welcome to The AI Show with John Koetsier. So, music is one of our oldest art forms, right? Art is creative. We think of people as being creative. We don’t necessarily think of machines as being creative or computers as being creative.
What about AI? Well, it might surprise you: I’ve actually been listening to AI-generated music for much of the last few days. I want to bring in the CEO of AiMi.fm, Edward Balassanian, to talk a little bit about a new app that he’s created that is AI-generated music.
Welcome!
Edward Balassanian: Thank you for having me.
John Koetsier: Excellent. Glad to have you here. Tell us what you built.
Edward Balassanian: Well, AiMi is as you described, an AI that generates music. We took a little bit of a different approach than the typical generative music systems out there. We’re not trying to replace human creativity with AiMi, instead we’re trying to augment it. Artists still create the beats if you will, the small phrases or words and sentences to tell a story, and AiMi puts those together to create a musical experience.
John Koetsier: Very very interesting. Talk to us about why you decided to build this.
Edward Balassanian: Well for me as a fan, I love listening to music and for me, music is usually an experience, more so than it is me wanting to listen to this song or that song. That’s part of it. The other part of it was I’ve been around DJs and producers a lot and I realized that a lot of what they do is tedious when they’re in a studio arranging music. The creative part of it is coming up with the beats, if you will, the little bits and pieces of music that go together to make what you hear in the music that AiMi’s producing.
So AiMi is really a combination of those two things: creating a musical experience for the listener and also giving artists away to take all the content they’ve created and provide them a new platform for expressing that.
John Koetsier: Well, I think we have to listen to a little bit of it. And as I mentioned off the top, I mean, I’ve been listening to AiMi.fm for big chunks of the last few days. And I actually found it really, really good to work to. So this is AiMi.fm. I’m going to put it right here.
[music plays]So right now most of what I hear from AiMi.fm is kind of electronic type music. Are you thinking of branching out beyond that at some point?
Edward Balassanian: Yeah. We chose electronica for a couple of reasons. One, that genre specifically has perfected the art of long form music. Most music that you hear today, pop, rock is geared around back in the radio days where songs had to be three minutes in length. Electronica grew up in house parties where you’d play music for hours at a time. So the producers and DJs have techniques that keep you engaged over long periods of time in a way that other genres don’t.
John Koetsier: Yeah.
Edward Balassanian: So we started with that genre, but one of the really exciting things about AiMi is later this summer, we’re going to announce our Artist Marketplace. In addition to … I think I just announced it actually.
John Koetsier: I think you did!
Edward Balassanian: Oops. Well, anyhow, we’re going to be announcing artists that we’re working with later this summer, and they’ll release artist packs. And you’ll be able to go into AiMi, listen to our background music, or you’ll be able to go pick your favorite artist and AiMi will play their beats with an AI trained by that artist.
John Koetsier: That is super interesting because you can imagine each artist has their own unique sound, what they do more of than somebody else, and now you’re actually train an AI to be generative based on that source work .
Edward Balassanian: That’s right. And we pick their creative content, so it’s really empowering for an artist as opposed to compromising their creativity and their artistry.
John Koetsier: Very very cool. So what AI technologies did you use?
Edward Balassanian: Well, we’re specifically using TensorFlow for the exercising of the model, but we have some proprietary tools for how we train the model. One of the things that we’ve been successful at doing is figuring out how to train AiMi so it can play long form electronica without getting into copyright issues that are kind of plaguing the AI world right now. So we feel like we’ve made some big steps in that regard.
We’re also able to train AiMi at an order of magnitude better than just feeding in song by song and teaching a system. Other systems out there right now, for example, require a lot of processing to succeed. AiMi runs on this, so you get your own private AI on your phone when AiMi’s playing, and that’s kind of a departure from the traditional systems.
John Koetsier: So that’s really interesting because you’re actually using my cycles, my CPU cycles to run it. It’s local. It’s on the edge, if you want to put it that way. What made you decide to use that model?
Edward Balassanian: Well, a couple of things. One, we wanted this music to be played for you, not at you. And having the AI local and the machine learning local is going to give us the ability to do real-time adaptation and personalization of the music. We’re just beginning to scratch the surface of that with what we have today. So that’s one of the reasons why we chose that.
The other one is, as you just said, all the processing happens on the edge. So we’re massively scalable because adding a new one of these costs us nothing
John Koetsier: As long as the cost of customer acquisition is nothing. Yes, I agree with you. Interesting. Now I notice as I use it there’s occasionally when i start up there’s almost like a loading or it starts playing but there’s a loading of something. What’s happening in the background while it’s loading, is it loading new intelligence ?
Edward Balassanian: Well, there’s two things happening. One, we’re downloading beats, so we’re constantly updating our beat library. And just to repeat again, a beat is essentially 4 to 16 bars of music. So you can think of it as four beats all the way up to 16 beats. AiMi is taking those, layering eight layers of that content, figuring out how to match the pitches, how to match the rates, dynamically changing pitches and rates, and then mastering it in real time. So all those beats that come down are a part of the library that AiMi uses to create the music on the fly.
John Koetsier: So that’s interesting. And I want to go back to something you said earlier, because you talked about being able to train AiMi with much less data than some other models or some other technologies. And I heard about that recently from Intel, I did an interview with them and their data training sets are very small compared to the hugeamounts of data that some people need. How are you able to do it?
Edward Balassanian: Well, the technology is a little proprietary, or I shouldn’t say little, it is proprietary right now, but I can talk to the general theme of it, and that’s instead of trying to teach AiMi how to pattern itself around existing music, we teach AiMi to learn basically listening to itself and comparing it to what it’s played, or what an artist might’ve played. And that gives us the ability to see how close we are and then have AiMi sort of get better and better at it.
Patterning itself around a specific artist’s style. That’s also the mechanism that we’re going to use to allow an artist to have their own style imparted into their artist pack.
John Koetsier: Interesting, so AiMi is learning from itself. Yeah, very interesting. What challenges or problems are you running into as you’re building out AiMi?
Edward Balassanian: Well, the first thing was finding the right team. And you know, I’ve been the founder of multiple companies, and I have to tell you, I’ve learned luck is a big part of success in any startup, and I just happen to have an amazing team right now. They’re all gifted musicians, they not only love music, but they’re also computer scientists. Several of them have PhDs in generative music or topics related to generative music, and we love what we’re doing. So I think the first step was finding a team. It took me a couple of years really to find the team. The idea is not a year old, but the team is. That was the biggest first step. The second step was really getting AiMi to sound good enough to release it. You know, we had some early versions that played music, but we had a rule that until you hit play and you’re delighted, we’re not going to ship it. So we feel like we’re there now. We’ve got a lot more to do. But that was really the first step was getting to the point where an average person hitting play would go, wow, that is really good!
John Koetsier: So either you weren’t happy with a minimum viable product or your minimum viable product was exactly that, that it sounded good on the first play. And have to tell you, it’s not the first AI-generated music startup that I’ve been pitched by, or the first technology that I’ve tried. And I recently tried something, I think it was only a month ago or something like that, and it was something where you could add a few elements and kind of customize it to what you’re going to do. And we should get back to that by the way because that’s something that I’m sure you’ll do at some point, I don’t know, but I could not make anything on this app that I thought, hey, that’s cool. I couldn’t make anything that even approached where I could make in something in GarageBand or something like that, and so I dumped it after a couple tries or something like that .
Edward Balassanian: You bring up a really good point… control. We actually went back and forth on this quite a bit and we realized that trying to give the user control was a bit of a cop out. If we’re not playing good music, then the knee jerk reaction is let’s let the user try to make it into good music. So we decided to go very genre specific and try to make the best application in this specific genre so that people who are looking for this kind of music will love it. If you’re looking for rock and roll, you’re not gonna like AiMi, but we didn’t try to make AiMi give you a control that you could change into rock and roll.
John Koetsier: And you started in the right place. I mean, if you start with rock and roll, multiple instruments, much more challenging, right? A narrative structure of the music which you don’t necessarily have in electronica. So I think it was a good choice. Now you’ve just launched, I think you launched a couple days ago, what’s the response been so far?
Edward Balassanian: Well look, we didn’t have grand expectations of having a hundred thousand downloads in the first day, but I think so far the reaction, the reception has been really good. We’ve had only five-star reviews in the app store. I can’t honestly tell you if those are all friends and family, they might well be. But really, I think the big traction is going to come when we start releasing our artist packs because one of the really effective aspects of the artists is they’re really good at getting their fans excited, and if they’re able to make money on Aimee, which they will be, we hope that they’ll be a really effective way for us to get users in adopting the platform.
John Koetsier: So that’s a super good segue. What’s the monetization model?
Edward Balassanian: Well, AiMi is a subscription model. So you get 30 minutes free a day, and then if you want to get unlimited playback, it’s $5.99 a month and we’ll release pricing for the artist marketplace later, but it’ll be higher. But that’ll give you access to all of our artists content and the artists share in the revenue stream that we generate, the subscription revenue stream that we generate.
John Koetsier: Interesting. I had a thought about that monetization model and I thought it was on the one hand under priced, and on the other hand, overpriced. I mean, because although on one hand it’s COVID-19 times right now, I’m not in an office right now, but I have been in office and you get in there and somebody plays some music, it’s a Spotify list or something like that, and invariably there’s stuff you can’t work to, you just can’t work to. You know, it’s supposed to be something playing in a club or it’s pop music for Idon’t know, 13-year-olds or something like that, you can’t work. And I felt like I could really work to the music that AiMi.fm was playing. It was something that was nice, it was something that was great, it focused me rather than distracting me. And I thought, hey, for an office setting, assuming we ever get back into offices, this would be an amazing thing and you should charge more, you should price it higher. And I thought for personal, I don’t know, maybe it was a bit too much. Your thoughts?
Edward Balassanian: Well, you’ve brought up two points that are really smart. One is the difference between foreground and background music. So a lot of pop and songs are meant to be in the foreground, which is why I personally find it hard to study to them or to work to them. They’re designed to grab your attention. We decided to go more with background music. Now, look, if you go up to energy 10 it’s a little hard to ignore, granted, but energy one through five I think is more in the background for you. The second point you brought up was more of a public setting or commercial settings for this. We have some exciting stuff to offer later this year around commercial use of AiMi and just to give you a hint, the same way that we can adapt AiMi to your personal tastes, we can adapt AiMi to a commercial or a public setting. And where we really feel like that whole industry is ripe for disruption right now. I mean, the way that music is pumped into public settings is archaic. Gestapos are sent out to police people whether they’re using it or not.
John Koetsier: Yes.
Edward Balassanian: I think we’re going to make some really big contributions there.
John Koetsier: So you make a great point. We didn’t talk about that earlier and I’ll just bring this in, I’m playing your music again but there’s an energy setting on there I’ll try and find it on my camera, there we go, and I can pump up that energy and I think it’ll take a bit of time here but I can pump up that energy to make… I don’t know, what happens to the music when I do that?
Edward Balassanian: Well, there’s composition changes. We change the compositional rules that guide us in terms of how we compose the music, and we also choose loops that are more suited for that energy level.
John Koetsier: Okay, cool. So what’s the next challenge or problem that you need to solve? I mean, you talked about the artist’s network that you’re bringing out, but in terms of the AI, what’s the next challenge to solve?
Edward Balassanian: Well, we really want to take the personalization to the next step. We really want this to be John’s music and Edward’s music, despite the fact that it’s an app that we all can share. And the same thing applies to the artists. We want the artists to really get a platform that they’re proud of, they don’t feel like they’re being compromised, and that their creativity is really flourishing on the platform. And that’s all going to happen with AI.
John Koetsier: Interesting. So project yourself out three to five years something like that, where do you see AiMi.fm?
Edward Balassanian: Well, I had mentioned earlier that I’ve started a bunch of companies and one thing I’ve yet to succeed at is creating a company that stands the test of time. An d this company feels like it can, it’s got a product that I think people will love, I think the monetization strategy works. I think the market is ready for it, both in terms of the artist communities that we can get engaged and the consumers. So in three to five years, I don’t see us replacing pop or the labels creating music, but I do see us as another platform for both artists and consumers to appreciate music.
John Koetsier: I also wonder, and maybe this would be kind of unprecedented ground, by wonder if something like this could be available inside of Spotify, or inside of an Apple iTunes, or something like that. That could potentially be super interesting as well.
Edward Balassanian: Yeah, and that’s part of the reason why we priced it lower than the Spotifies and Apple musics is because we want you to have this in addition to those services. And we would love to do partnerships with companies like that and have a channel on a Spotify, or iTunes, or Sonos, or whatever.
John Koetsier: Very good. Well Edward thank you so much for joining us, I really do appreciate your time.
Edward Balassanian: My pleasure, thank you.
John Koetsier: For everybody else, thank you for joining us on The AI Show as well. Whatever platform you’re on, like, subscribe, share, comment. If you’re listening on the podcast later on, please rate it, review it, that would be a massive help. Until next time, this is John Koetsier with The AI Show.