24-02-2016, 13:24 #1
[EN] Spotify ditches AWS and Hadoop for Google
Music-streaming service quits data centre biz.
Feb 24, 2016
Spotify has ditched its data centre business, and confirmed plans to dance off into the sunset with Google, having inked an infrastructure deal with the search and ad giant.
The agreement—financial terms of which were kept secret—represents a big shift in strategy for Sweden-based streaming service Spotify. Up until now, the company had operated its own data centres, while using a cloud storage service provided by Amazon Web Services.
Spotify said it was now changing direction because of Google's data stack.
"Building on Google's data infrastructure provides us with a significant advantage where it matters the most," gushed the music-streaming company's engineering and infrastructure veep Nicholas Harteau.
Google added that Spotify had split its migration to Cloud Platform into a services track and a data track, and was in the process of moving some of its services away from its homegrown infrastructure over to Google's public cloud.
Additionally, Spotify is using Google storage, and networking services. On the data front, Spotify is pulling in an "entirely new technology stack" by ditching Hadoop, MapReduce, Hive, and a number of other tools in favour of Google's offerings.
As noted by the Wall Street Journal, Amazon had previously gloated about Spotify's reliance on AWS. But Spotify just muted the volume on that particular deal.
24-02-2016, 15:56 #2
Feb. 9, 2016
Spotify, the market leader in subscription music streaming, is now seeking to borrow $500M from investors through a convertible bond offering — just eight months after it raised the same amount by selling equity at a valuation of $8.5bn.
A velha fórmula perigosa do sem dinheiro mas com clientes se amancebando com o sem clientes mas com dinheiro
Esse casamento (Beta) vai calar por uns tempos os 17 bilhões de maldosos que dizem que a única forma do Google Cloud aparecer no noticiário é reduzindo preços, factóide sem consequencias financeiras porque o GC não tem clientes.
Última edição por 5ms; 24-02-2016 às 15:59.
24-02-2016, 16:15 #3
Spotify chooses Google Cloud Platform to power data infrastructure
Tuesday, February 23, 2016
It’s not every day you move a 75 million+ user company from a home-grown infrastructure to the cloud. But if you use Spotify, more and more of your musical experience will be delivered by Google Cloud Platform over the coming weeks and months — we’re partnering on an ambitious project to move Spotify’s backend into GCP.
Spotify aims to make music special for everyone. Today, the company hosts more than 2 billion playlists and gives consumers access to more than 30 million songs. Users can search for music across any device by artist, album, genre, playlist or record label, while features like Discover Weekly suggest personalized playlists for millions of people around the world.
While Spotify had engineers running its core infrastructure and buying or leasing data-center space, PC hardware and networking gear to provide a seamless experience for users — time and again it asked whether the tradeoff of resources that could otherwise focus on innovative features and software, was worth it.
Recently Spotify decided it didn’t want to be in the data center business, and chose Cloud Platform over the public cloud competition after careful review and testing. The company split their migration to Cloud Platform into two streams: a services track and a data track. Spotify runs their products on a multitude of tiny microservices, several of which are now being moved from on premise data centers into Google’s cloud using our Cloud Storage, Compute Engine and other products.
With Compute Engine, teams can rely on consistent performance from ultra high IOPS SSD and local SSD storage capabilities. And with autoscaling, they can build resilient and cost-efficient applications that use just the right amount of resources necessary at any given time. For storage, Spotify is now implementing Google Cloud Datastore and Google Cloud Bigtable. This rich fabric of storage services lets engineers work on complex back end logic, instead of focusing on how to store the data and maintain databases. Spotify is also deploying Google’s Cloud Networking services, such as Direct Peering, Cloud VPN and Cloud Router, to transfer petabytes of data. This results in a fast, reliable and secure experience for users around the globe.
On the data side of things, the company is adopting an entirely new technology stack. This includes moving from Hadoop, MapReduce, Hive and a series of home-grown dashboarding tools, to adopting the latest in data processing tools, including Google Cloud Pub/Sub, Google Cloud Dataflow, Google BigQuery, and Google Cloud Dataproc.
With BigQuery and Cloud Dataproc, data teams can run complex queries and get answers in a minute or two, rather than hours. This lets Spotify perform more frequent in-depth, interactive analysis, guiding product development, feature testing and more intelligent user-facing features. To gather and forward all events to its ecosystem, Spotify is using Cloud Pub/Sub, Google’s global service for messaging and streaming data. This gives teams the ability to process hundreds of thousands of messages per second, in a reliable no-ops manner. And to power its ETL workloads, Spotify is deploying Cloud Dataflow, Google’s data processing service. This lets the company rely on a single cloud-based managed service for both batch and stream processing.
What makes us most excited to work with Spotify is their company-wide focus on forward-looking user experiences. Now that they’ve begun using Google Cloud Platform, we can’t wait to see what Spotify builds next.
Join us for the GCP NEXT 2016 opening keynote, where we’ll feature a talk from Nicholas Harteau, VP of Engineering and Infrastructure at Spotify. You can also attend Spotify-led technical sessions where you can learn more about how they’re deploying Google Cloud BigQuery and Dataflow.
- Posted by Guillaume Leygues, Lead Sales Engineer, Google Cloud Platform
More and more of your musical experience will be delivered by Google
Última edição por 5ms; 24-02-2016 às 16:21.
24-02-2016, 17:07 #4
Spotify Is Seeking More Than $500 Million in Convertible DebtAdam Ewing
January 27, 2016
Spotify Ltd., the world’s largest paid-music service, is trying to raise more than $500 million with a loan that could be converted into stock if the company goes public, according to a person familiar with the company’s plans who declined to be identified.
Spotify, which said in June that it had more than 20 million subscribers, ...
(curioso número: Spotify finished 2014 with 15 million subscribers globally along with 45 million monthly users of its free, advertising-supported service. Subscription revenue accounted for 90 percent of revenue, the same percentage achieved in 2013. After launches in the Philippines, Canada and Brazil in 2014, the service is available in 58 countries.)
... hired Goldman Sachs Group Inc. as its adviser, said two people, who asked not to be named discussing the private effort and wouldn’t discuss terms such as valuation. Nordea Bank AB is also an adviser, according to the Swedish daily Svenska Dagbladet, which reported on the fundraising on Wednesday. The people didn’t discuss valuation. Spotify, based in Stockholm, was valued at about $8 billion when it raised money last year.
The loans would give Spotify flexibility to grow and make acquisitions as the market for streaming music consolidates. A planned initial public offering is between one and three years away, according to one of the people. Spotify bought two companies this month aimed at making it easier to discover and share music with friends.
Spotify is also facing new competition from big players. Apple Inc.’s Apple Music has amassed more than 10 million subscribers since summer, while Pandora, the world’s largest online-radio service, is planning an on-demand music service.
Other competitors, such as Deezer SA and SoundCloud, have either raised money or licensed music from record labels in the past month. Record labels have heralded on-demand streaming as the future of the music business, in the hopes that enough people will pay for monthly subscriptions to end nearly two decades of declining sales.
Última edição por 5ms; 24-02-2016 às 17:20.
24-02-2016, 17:11 #5
Does Spotify Stimulate or Depress Music Sales?
Spotify has compromised the streaming/revenue model before the industry could even establish it.
Streaming Reaches Flood Stage: Does Spotify Stimulate or Depress Music Sales?
Luis Aguiar, Joel Waldfogel
NBER Working Paper No. 21653
Issued in October 2015
Streaming music services have exploded in popularity in the past few years, variously raising optimism and concern about their impacts on recorded music revenue. On the one hand, streaming services allow sellers to engage in bundling with the promise of increasing revenues, profits, and consumer surplus. Successful bundling would indeed translate some of the interest in music not generating revenue through individual track sales - unpaid consumption and deadweight loss - into willingness to pay for the bundled offering. On the other hand, streaming may displace traditional individual track sales. Even if they displace sales, streams may however still raise overall revenue if the streaming payment is large enough in relation to the extent of sales displacement. We make use of the growth in Spotify use during the years 2013-2015 to measure its impact on unpaid consumption and on the sales of recorded music. We find that Spotify use displaces permanent downloads. In particular, 137 Spotify streams appear to reduce track sales by 1 unit. Consistent with the existing literature, our analysis also shows that Spotify displaces music piracy. Given the current industry’s revenue from track sales ($0.82 per sale) and the average payment received per stream ($0.007 per stream), our sales displacement estimates show that the losses from displaced sales are roughly outweighed by the gains in streaming revenue. In other words, our analysis shows that interactive streaming appears to be revenue-neutral for the recorded music industry.
07-03-2016, 22:39 #6
How Much Is Spotify Paying for Google Cloud?
March 7, 2016
Two weeks ago, Spotify announced it was migrating from its own datacentres to Google Cloud Platform.
This is a huge win from Google because Spotify is the first major service running at huge scale that is deploying across many of its cloud products (and talking about it). We all know that Snapchat has been running on Google for a while, but since it is primarily on App Engine, Google needed a credible use case for its other services. Now it has one. This is similar to AWS’s Netflix.
Spotify is gradually revealing the numbers behind its decision to move to the cloud, further strengthening the case for “why Google?”. However, they are missing a major evaluation criteria: cost.
Thankfully, Google’s pricing is very transparent and simple to work with, so we can use Spotify’s blog posts to calculate some list pricing for what Spotify might be paying Google.
So far, event delivery is the only service we have real numbers for. Spotify was previously using Kafka in its own data centres but has decided to move to Google Pub/Sub.
Google’s Pub/Sub pricing is based on 3 key variables:
- Operations. Every API call is an operation and one “event” will consume at least 3 API calls: publish, push/pull to the consumer and an acknowledge. This assumes the message is 64KB or less.
- Networking. If you are using Pub/Sub within a single region then there’s no networking cost, but if there’s communication to another region then you pay the standard networking fees.
- Storage. Messages are stored until acknowledged, which can be for up to 7 days.
Based on Spotify’s blog post, we can extract the following numbers:
- Operations. Spotify’s average production workload is 700,000 events per second. During their test, they used a number of 2,000,000 events per second to stress the system and allow for future growth, but it’s fairer to use their actual quoted production workload figures.
- Networking. Spotify’s test was in a single zone so the networking costs are $0. However, if we assume they deploy across multiple regions (as they should for proper failover/disaster recovery) then the ingress is free but the push call to a consumer in the other region will incurr fees. Spotify noted a sustained 1Gbps of traffic for 2,000,000 events per second, or 0.125 GB per second. If all those messages have a single separate-region consumer, that means 0.0000000625 GB per message. With regional egress at $0.08 per GB (just simplifying on the best pricing tier) that’s $0.000000005 per message.
- Storage. This is harder to calculate because Spotify saw around 20 sec of end to end latency, making it difficult to know how much storage is used. This would also change depending on maintenance, downtime, consumer latency, etc. For simplicity we’ll just assume there was zero storage.
How much Spotify is paying Google for Pub/Sub
Taking the above calculations, we can come to the following conclusions:
- 700,000 events per second = 1,874,880,000,000 events per month
- = 5,624,640,000,000 API calls per month (3 API calls per event)
- = $300 for the first 1,750,000,000 API calls
- = $0.05 per 1,000,000 API calls after that
- = (5,624,640,000,000–1,750,000,000) = 5,622,890,000,000
- = (5,622,890,000,000/1,000,000) = 5,622,890
- = 5,622,890 * $0.05
- = $281,144.50 per month
$0 if all within a single region but if we assume 1 secondary region consumer:
- 1,874,880,000,000 messages * 0.0000000625 GB
- = 117,180 GBP
- = 117,180 * $0.08
- = $9,374.40
This would change if there were multiple consumers across multiple regions because Pub/Sub automatically balances across all consumers. E.g. you could have 1 consumer in the “source” region and 1 in the secondary region, and so you might only pay for half of the messages to be consumed in that secondary region.
$281,144.50 + $9,374.40
Is Spotify really paying $290,518/month?
Of course, for such a high profile customer and for the priviledge of a case study, we can assume Spotify has some heavy discounts. Cost is important but using public cloud and especially managed services like pub/sub, the biggest benefit is in ease of deployment and not having to manage/scale everything.
Also remember that this is just one part of Spotify’s product. There’s all the other Google products that we have still to learn about — BigQuery, Dataflow, Compute Engine and in particular, the networking costs for streaming all their media. I wonder if they will run their own CDN like Netflix does, or if they’re buying that from Google too.
It’ll be interesting to see how all the other components add up!