Wednesday night a few million of you decided to clog up the internet. Several computers caught fire, nerves were frayed and big hosting companies made promised that were broken. Even the Ch4 DB site went down and they have huge clusters. You lot should be ashamed.
However we managed 3 of the top WORLD twitter trends, the “Derren Brown Lottery Prediction” site shot in to the top 20 wordpress sites within a few hours and we even made Australian news (Hello Oz! *waves* We love you but we’re not giving Tim Minchin back so forget about it!).
We gained an extra 10k followers on facebook and similar amounts on twitter overnight and a whole bunch decided to email asking for “how it was done”. Now… seriously? and even more told us how DB has OBVIOUSLY done it and that we are “stupid”. Well we shall see…!
And so the several hosting companies we’ve gone to have said they have “never seen anything like it” but we’re working 24/7 on a solution to the problem of CPU usage bottling out the site.
I’m sure some genius amongst you all out there will come forward and tell us how to stop wordpress being a gargantuan memory hog or to use a hosting company that can handle 100k+ people all bombarding the site in one hour and we are very happy for you to contact us so that Abeo can finally get some sleep. Your suggestions are very welcome – from cloud hosting to grid servers we’ve tried them all but maybe you know better.
Thanks for all the lovely support too – I want to lick and touch you all.
"
yes, happy for you to email me with any solutions!
abeo@derrenbrownart.com
We’re all naughty little tinkers aren’t we
Sorry Abeo and P!!
Let the licking commence.
Why not publish the “hot” content as static HTML and leave wordpress for the cold/old pages which are getting low hits.
It wasn’t us, it was all of the ‘New’ fans. They dont understand, they weren’t there in the beginning they dont know what its like!!!
Oh and the licking and touching sounds delightful.
Great show on Wednesday, really looking forward to tonight’s ‘reveal’.
[Tech bit] You’re probably using it already but try a WordPress plugin called WP Super Cache. It creates static HTML pages from WordPress’ PHP pages which should reduce server load. More information here: http://wordpress.org/extend/plugins/wp-super-cache/
Hi guys,
I’ll email this to Abeo as well, but are you using a caching plugin for WordPress, such as WP-Super-Cache? There are also some MySQL config optimisations you can make, which could help. Please see this article: http://www.codinghorror.com/blog/archives/001105.html
Hope it helps! Can’t wait to see tonight’s show.
All the best,
Sam
Mmmm, touchy touchy lick lick.
Valiant efforts, but clearly no match for the colossal fan-base!
I’ve been amazed by how 100% certain people are that their ridiculous theory is the correct one.
There are a number of people who have basically been saying “You are all wrong. It was clearly done by -XYZ-. WHY WON’T ANYONE ACKNOWLEDGE MY DEDUCTIVE BRILLIANCE” over and over again.
The aftermath has been almost as entertaining as the trick itself.
Can’t wait to see how it was really done. I hope EVERYONE is wrong.
Phillis, you know you love us!
DB’s made it into Norwegian press as well, even though they don’t show his shows here anymore
http://www.dagbladet.no/2009/09/10/nyheter/utenriks/lotto/8038936/
(beware of hole-tempting!)
Aha! So you’re the one holding our Tim hostage! Let him go – he’s got a Oz concert tour coming up soon and I’d hate for all us atheists to turn up and find out they’ve rescheduled it to an Andre Rieu concert. Have pity Derren!
We actually broke the internet… but the rampant speculation has been great fun, I, of course, know exactly how it was done but I’m saying nothing.
(Actually, I’ve not got a clue… we’ll see!)
When are you going to finally admit that you are a jedi?
We have tried WP Super Cache yesterday which took the blog down rather spectacularly, can’t seem to get it to work on our set up
(If your search the wordpress forums it seems an awful lot of people have the same issue with it)
ooh sounds promising!
Yes simply email me next week’s loto numbers and I will be right over to fix everything.
But seriously I think that is the issue a lot of people have had with wordpress and is just a disadvantage with dynamically generated pages. Try something like eAccelerator.
@ Abeo
Some people eh? So God Damn rude!
You’re all stupid?, wtf! Yes of course, that’s why you’re all part of something that has managed to dominate so quickly in such a tiny amount of time!
And personally.. I’d say it’s a bit stupid being insulting to you guys anyway… never know what subliminal instruction D could tell you to reply to those sorts of people with… I’ll be good, promise! Lol.
But yessss, extremely impressive. Bravo! No doubt w’ere going to see this possibly happen again perhaps another 3 – 4 times… haha.
Tell you what, I email you the name of a company that I was considering using for when I can eventually get my poop together…. may help, but don’t curse me if not
Peace People!
@Sam W: I’ve tried various SQL optimisation recommendations, but I’ll give those a go, thanks
Uhm .. some advice .. some great words … once written somewhere …
http://derrenbrown.co.uk/blog/2009/09/wrong/
That should make all things better ….. trust me … works wonders ….
If every one *knows* how it was done then why haven’t they done it and sold the secret to all the cheap fake illusionist out there?!
All you need do is send out a flock of carrier pigeons:
http://news.bbc.co.uk/1/hi/world/africa/8248056.stm
I don’t have a clue, and I don’t care how many posts it takes me to explain that.
I don’t mind the licking though, though licking upwards of 10k people might make your tongue a bit dry… just remember we were here first Phillis!
Yes, it was shown on Aussie TV. Channel 7′s “Today Tonight” linked the promo for your event with a story about a guy who they claim knew he’d won the lotto before he’d checked his ticket. He’d heard on the radio that Lotteries were trying to contact the winner and when he got home, there was a message from Lotteries on his answering machine. He returned the call and said “I haven’t won 2 milliion in lotto have I?” Spooky, for sure – I don’t know how he did it but I hope he uses his powers for good, not evil.
Today Tonight even described you as a psychic in one lead-up. I’m surprised they eventually hinted that you might be faking it. Channel 7 spend a great deal of time promoting psychics and ridiculing sceptics. You really need to pay them a visit and slap them around a little.
Here is my guess.
1. Camera work and split screen are involved, but not how we think.
2. When the draw starts, Derren watches the drop of the balls.
3. He says he has spent the last year determining likely outcomes from the first few seconds after the balls drop.
4. Once he puts his hand to his mouth, the entire screen is frozen (not just the left hand side).
5. Derren has perfected that pose over the last year (watch him check he is standing on the right spot).
6. Derren himself then walks to the stand!
7. The TV is a green screen to allow this to happen.
8. He quickly picks up the 6 balls (from 49 available) that he has calculated in 3. above and puts them in the tray in order.
9. He does this before the lottery numbers are drawn.
10. He walks back to the right hand side and resumes his pose.
11. The camera fades back in to the live picture, deliberately shaking throughout to disguise.
12. The left hand ball is slightly too high (error or deliberate?) and this is the exact moment the live pictures recommence.
13. The ‘second cameraman’ films all this happening and we will be shown those pictures tonight.
14. In the programme tonight Derren will give us all sorts of ‘probability’, ‘studying the balls for a year’, etc.
15. We will believe that he did actually predict the numbers and placed them in the tray himself before they were drawn.
16. He must have done as we will see it on the film from the ‘second camera’. And we will be amazed.
17. But actually that’s not how he did it after all, it’s just how he wants us to believe that’s how he did it!
18. And he really did it some other way (split screen with ‘second cameraman’ placing the balls after they were drawn?)
19. He certainly did not actually predict the numbers before they were drawn.
20. So the real illusion is tonight when he tricks us into believing he predicted the numbers.
You heard it here first.
You could try lighttpd (http://en.wikipedia.org/wiki/Lighttpd) instead of Apache. Updating a static html index page sounds like the best option until this swarm dies down. That’s what folks usually do when their site appears on Digg or some such, I believe.
I can’t wait for tonight’s reveal, it’s going to be fantastic.
I’ve been reading all the different opinions on how it was done, and they range from the stupid, to the impossible. The obvious ones seem far too obvious, easy, and not at all Derren’s style. However, I do think it’ll be a “Wow! Of course!” moment.
If it turns out to be a split-screen I’m going to cry, I’ve made a lot of promises to eat a lot of hats if it turns out to be a split-screen.
Also, give this a read – http://blog.thembid.com/2007/07/17/how-to-handle-thousands-of-users-with-your-blog/
I’m only 12. Im telling that you want to lick and touch me.
- No your not Dan, now queue up! – Phillis
I run a hosting company, I have sent an email to suggest a solution.
Would love to know the currently infrastructure layout of your hosting so I can provide a better plan, would also need to know the budget if possible. [you can send the details to my email david@smoothosting.co.uk]
Would really like to work closely with you to work on a solution.
I’m going to make a prediction on how it was done at 10:01pm tonight!
Glad you’re back. Missed you yesterday. Brilliant show on Wednesday (I was feeling sick myself). Can’t wait to see how it was done in tonights show.
Love the new look “old” site, by the way.
maybe lily tomlin was right…try 2 dixie cups & a string…
@nina
It wasn’t us, it was all of the ‘New’ fans. They dont understand, they weren’t there in the beginning they dont know what its like!!!
Oh and the licking and touching sounds delightful
your a fan of spaced to? =D (or just coincidence
)
(i also think simon pegg looks alot like derren brown in the “simons cam” on shaun of the dead, just me?)
You shouldn’t lick me, you don’t know where I’ve been.
No really, you have NO idea where I’ve been.
@flapjack- with you on that one!
I couldn’t access your page yesterday night …very disappointed with that. Do something about it!
You are also on several major Hong Kong newspapers:
http://news.mingpao.com/20090911/gza1.htm
http://hk.apple.nextmedia.com/template/apple/art_main.php?iss_id=20090911&sec_id=15335&subsec=15339&art_id=13196832
http://the-sun.on.cc/cnt/china_world/20090911/00423_001.html?pubdate=20090911
(They are in Chinese but you might be able to spot a certain familiar name here and there.)
Agrees with flap…we were here first Phillis!
Make a copy of the whole website, sharing the same database and with a javascript or a check made in another language, redirect people between them. Don’t know if it’s doable, just brainstorming
This is all so fantastic. I don’t know what is more outstanding, the fact that Mr. Brown was able to accomplish what he did, or the fact that he is going to tell people HOW he did it. DB eluded to having lotto numbers on his walls for a year, but that may have been a bit of miss direction. I’ll accept any answer he gives, even if it is “I’m Just That Good”.
Thanks for bringing the child-like wonder that has escaped me in my adult years, Derren.
As to the traffic issue- Unless DB is going to predict the lotto more often, I don’t think there will be such a massive influx of people to that extent… Most of those hits were no doubt fueled more so by greed than general curiosity.
What process is causing the CPU load? (Is it your DMBS, HTTP server, application container, or otherwise?) Put a proxy (such as squid) upstream of your HTTP server, preferably on a different machine altogether; cluster and replicate your DBMS to reduce query load; get rid of unnecessary stuff on your layout such as the “recent comments” section.
Running on something like AWS would be a last resort, and not an overnight conversion, but I presume that there are AMIs already available with the Squid/Apache/PHP/MySQL config that I presume you want and you should be able to scale on demand.
Why would anyone doubt the brilliance that is Derren?
The man is a genius.
Licking and touching – hm promises,promises……….
xx
you need to be using caching if you are having CPU problems… Oh I see that’s been recommended, still, that’s the solution, you just need to work out how to get it working :¬/
ok The Boss maybe Derren and he maybe right but how about this. Thermal paper across each ball (remember raised squares holding the numbers) and tiny lasers inside and behind the shells of the balls. If you remember Derren talked for quite some time after the draw before turning around the balls ,this was so the images could be formed. I used to work for a large printer company for 17 years. If I had to do this trick believe me it is possible.
Well, that does not need to be a problem ofcourse, your very last line … You know the grocery store that is closest to your place? Let’s say around 22.00? At the entrance? I might even tell you how to fix the server trouble .. if you are nice.
Ehehe ….
Btw: it looks as if every time you post something here the whole system becomes instable ..
Good luck with the problems !(I can’t stop having the feeling that you called this upon this server by posting that ‘if all goes wrong in your life” post ….but erm … )
What makes me laugh the most are the amount of people saying that Derren has ‘failed’ or ‘been busted’ as they ‘know’ how his trick was done.
He’s made national press 2 days running (full page in the Daily Star today), the internet has gone into meltdown over it, and virtually every media outlet in some form or other is talking about the trick. He has pretty much guaranteed everyone will be watching his show tonight.
If that is Derren ‘failing’, I’d hate to see him when he ‘succeeded’…
Paul B xx
Well that’s your first problem, wp-super-cache should let you handle 50x the traffic you can without it. A shame you haven’t been able to get it to work, there must be a way. If you still can’t get it to work, try this one instead:
http://www.w3-edge.com/wordpress-plugins/w3-total-cache/
You need to get something up by tonight, or you’ll be overloaded again (and if you’re not on a dedicated server, probably overloaded even with it).
Also try moving and pointing to all the files/images you can onto Amazon S3/cloudfront so it reduces the amount of httpd connections. Combine as many files/images as possible for the same reason.
You might want to ask channel 4 if they can recommend a solution for you. Other than adding more servers (although as its WP it might not support multiple node points) there is not much you can do. WordPress should have gigabit bandwidth, and servers reserved for a thing like this so get them to flip you over to one of them.
The actual prediction show was possibly a simple case of clever a TV special fx and camera trickery/split screen…but this WASN’T the trick!
))
WAIT FOR THE SHOW TONIGHT PEOPLE!!!
ok…
1st – it’s a TV programme – so it’s been written (and re-written), edited, rehearsed etc. So ANYONE who thinks this is for real, please go a way..
next – the whole Derren Brown ‘thing’ (if you haven’t already noticed) is characterized by the following.. He does something, normally amazing/fantastically wonderful/mind blowing piece of trickery…
This familiar format is likely to follow in tonights show….. he’ll demonstrate that he did indeed guess the numbers BEFORE the actually lottery draw anbd provided us with the evidence. THIS WILL BE THE REAL ‘MAGIC TRICK’
Have you tried CacheFly they’re supposed to be Digg proof, tho perhaps nothing can survive almost worldwide simultaneous TV coverage.
I think she’s ganna blow captain…
Have a sitdown, maybe a nice cup of tea, and then some thinking time.
Oh, and don’t forget the backup. Backup! Backup! What backup…
The licking and touching sounds very pleasant.
Try http://www.akamai.com/ for your troubles…
@Stuart/Abeo – Good call, I also can recommend the AWS route, that’s what we’ve just moved to for our competition platforms at work… a bare debian AMI can be spun up and configured with a tight, efficient LAMP stack in under a couple of hours, and once you’ve bundled your own AMIs you can spin up duplicate machines in under 4-5 mins as load dictates.
I’d recommend a dedicated NFS server instance, to handle the EBS mounts, one XL MySQL server, and 2-10 webheads as load requires. Each webhead maintains it’s own cache, and DNS-based load balancing shares everything out (and eliminates cross-server session woes).
Memcached would be my caching recommendation, but I’ve not used WP that much – this might be useful: http://ryan.wordpress.com/2005/12/23/memcached-backend/
</geek>
Well…..what a few days in the Derren Brown world! Blimey!!
Am soooo looking forward to 9pm tonight!! I shall be riveted to my TV screen & for once I’ll be saying “sod The Tudors!” on the other side….
Sorry, I can’t help with re to the blog, but I’m sure when you find a solution it’ll remain as fabulous as the people running it. (Ouch, how sucky does that sound?!
)
Good luck with everything!
LC x (been here a while…..
)
Do not give up on wp-supercache. Find out what’s going wrong and fix it. No popular WordPress blog can do without it.
Well how about if the video/images were distributed via p2p networking, such as the bittorrent protocall?
How can I watch tonight’s show in the United States? Is there someplace I’ll be able to see the entire program online? It doesn’t have to be live. It’s an hour-long special, right? Too long for YouTube unless it’s broken into six chunks.
Yeah, didn’t predict that, did you
Looking forward to the reveal (if such it is – well, looking forward to it anyway)
Have just noticed ‘Derren Brown’ is the number one search on Yahoo! today… looks like the events are really making a splash!
I’m so excited about tonight, but also to find out where the Science Museum drawing , and the kids on the bus will be slotted in.
Stephen Fry may be the person to ask about hosting – he seems to be the king of taking down the interwebs
I hope speculation matches the reality.
Trained Fleas, or thermal lazer written prediction balls pls.
Sorry!! Forgive us?
But I’m glad we get to keep Tim Minchin
(I have one of his shows on sky+ so I’m watching it now 
)
Hope all the website internetty rubbishh gets sorted out, but I’m afraid I can’t help out there…
Can’t wait for tonight- I convinced 2 people who were planning on going out tonight to stay in and watch. Me and my whole family will be sitting round to watch it with a friend and her family that we’ve known for 16 years. Tis very exciting for us!
Hi Abeo,
As the others have said you need to use caching to keep people from hitting the web and database servers in the first place. Have a look at solutions like Varnish running on cloud servers to sheild your main servers from 90%+ of the traffic, as surely nearly all of it is read-only.
HTH.
Gavin
Hey! I’m a new Derren Brown fan
I saw the advert for the events a few weeks ago, and I thought it looked cool, and what he did on Wednesday was mind-blowing!!!
I think that the people saying he “failed” or saying that it’s fake aren’t getting the point – He’s an entertainer – he entertained everyone on Wednesday and surely will entertain again tonight! We should just sit back and enjoy the awesome-ness!
Maybe worth having a quick look at this hosting company? http://www.liquidweb.com/select/
Well, DB what a stir you’ve created. Congratualtions! Am spreading the word and looking forward to tonight’s explanation.
good luck!
Yeah… we crushed the internet!
You do realise this has now become an anticlimax and poor Derren can NOW, NOT NEVER give a good enough answer!
why dont you just use a WP pluging to cache your pages ? and maybe tweak your DB and hosting company.!
http://mediatemple.net/ A very good host must help whith your traffic issues
Msg from Abeo: That’s who we were on and having the issues with!
I have this feeling, that after tonight’s show, we’re going to be left asking just as many questions as we were before. But I, for one, won’t mind. I love wonder, illusion etc, and sometimes I just don’t want that whole facade shattered.
Don’t berate me for saying this, but I wonder how many of you will feel much better for knowing how Derren did it, or afterwards still wish you didn’t know? Doesn’t anyone want to keep that element of mystery or of something unexplained, untouched?
Am I strange for feeling this way? Perhaps it’s the dreamy Pisces in me!
LC x
My theory for tonight’s show, where he is going to show you how to do the trick at home. In my opinion, this rules out camera tricks, laser etching and inkjet printers, etc…
He filmed 55,490 “reveal clips”, enough to guarantee getting 5 of out 6 balls correct. He just got lucky on the 6th.
The programme ends less than 60 seconds after the last ball is drawn – and is rushed for very good reason, to limit the length of each reveal clip so that they can be filmed in approx. 6 months. 55490 clips would take approx 920 hours to film.
The teaser for tonight’s show is a snowflake, this is a reference to the set of clips, each different from the last.
To do trick at home, Derren will upload clips to YouTube, each tagged with first 5 numbers in sequential order.
I know exactly how you can solve your problems with overloaded servers.
1) predict the lottery numbers.
2) buy a ticket
3) use the winnings to buy loads of servers.
simple. surprised you never thought of that one.
Phillis and Abeo,
Thank you for all that you have done and I’m not just talking about in the last 72 hours ( you should be promoted to semi-god for that). I think I can say we’re all sorry that we caused you some sleep less nights (everyone nod and look sorry).
Wow – Grea Job one by Derren on Wednesdy, just before ths shwo I made a Comedy PArody video which now has over 27,000 hit son youtube here: http://www.youtube.com/watch?v=aQYPXwnOjWM ENJOY and good luck to Derren tonight!
Phyllis Falsus eh?
Very sneaky… lol.
And to think… I was trying to figure it out numerically to begin with! I think I got most of them right… but the F & P threw me off.
Hahaha.
You funny sod’s
Peace people!
@AndyD
My face dropped when I heard Derren Brown’s name on OUR television. I mean, we don’t even get his shows. And I was so excited, I thought they’d be full of praise and awe. But oh, you’re right, Today Tonight does what it does best, blow things out of proportion. Naw, they made Derren out to be some evil trickster! Silly!
*waves back from Oz*….
You’re always welcome to come over for a visit…. I won’t even insist on licking…. touching maybe….
we can watch my Tim Minchin DVD together…. *s*
To all the people who got angry that they never got a hard explanation for winning the lottery:
http://cheezburger.com/View.aspx?aid=2624866560
Enjoy Friday, guys!
I must say that the explanation given did confuse me greatly… Surely the numbers would all come out at around 24/25… and even if they were ordered first they would come out at 7, 14, 21, 28, 35, 42…
+1000 awesome points for mentioning Tim Minchin! (Seeing him live in Edinburgh this october w00t)
(Oh, and I love the DB smileys)