Today i hit some new milestones in Distributed Computing, which i have been dabbling in for the last few years on and off on a part time basis depending on how my electricity and bandwidth i can allocate.
Majestic-12 is working towards creation of a World Wide Web search engine based on concepts of distributing workload in a similar fashion achieved by successful projects such as SETI@home and distributed.net. You can read more details on the project here. Below you can see recent news highlights, but for full details its best to check our forum.
I'm pretty impressed to have managed to crawl this many URL's in the last few years. You'd never be able to do this kind of web surfing normally if you tried. My target to reach at the moment is about 1,000,000,000 urls (1 billion urls)
I'm averaging about 1.2 - 1.5 million urls a day at present so it will probably take me another 1 1/2 years to do this. Thankfully I have the time and the computing resources available.
If you want to find out more or join the project you can download the software (the crawling bot) and get help from their website at http://www.majestic12.co.uk/
I joined this project recently and found out about it from AMD Users. Since a child i've always had a deep love for anything to do with Astronomy and wished I'd been able to pursue more of an interest in it when i was younger. Unfortunately due to the fact that I live in the suburbs of a large city, i tend to find that star gazing is nowhere even remotely near the beauty of the stars and the universe when you're somewhere truly dark and unspoilt by city lights. Maybe one day i will get my dream. Anyway back to the project, it's better explained by this segment from their About Page and this video:
Have a computer? Want to help astronomers make awesome discoveries and understand our Universe? theSkyNet needs you!
Your computer is bored. It has spare processing power nearly all of the time that could be used to do something cool. So why not let it?
The Science Bit
By connecting 100s and 1000s of computers together through the Internet, it’s possible to simulate a single machine capable of doing some pretty amazing stuff.
theSkyNet is a community computing project dedicated to radio astronomy. Radio astronomers use radio telescopes (of course) to observe the Universe at radio wavelengths (still with me?). All day, every day, signals from distant galaxies, stars and other cosmic bits and pieces arrive at the Earth in the form of radio waves. Once detected by a radio telescope the signal is processed by computers and used by scientists to support a theory or inspire a new one.
The Computer Bit
When you join theSkyNet your computer will help radio astronomers process information and answer some of the big questions we have about the Universe.
As a part of theSkyNet community your computer will be called upon to process small packets of data, but you wont even notice it’s going on. The key to theSkyNet is to have lots of computers connected, with each doing only a little, but it all adding up to a lot.
At the heart of theSkyNet is this website, theSkyNet.org where you’ll find alliances you’ve joined stack up against others. The more data you and your alliances process, the more status you’ll have within theSkyNet community.
But that’s not it, because as theSkyNet project evolves we’ll be adding more features for you to explore. In the pipeline we have visualisation tools to help you understand the data you’re processing and even an opportunity to help identify and catalogue radio wave sources.
The Guts of theSkyNet
theSkyNet is powered by some clever software called Nereus. Data collected by one of several radio telescopes is sent to your computer as a small data packet ready for processing by the Nereus client. Once processing of the data packet has taken place it is sent back and the process begins all over again. By repeating this process across thousands of computers, it is possible to simulate a single powerful machine capable of doing real and relevant scientific research.
You can join The SkyNet and learn more by visiting their website at http://www.theskynet.org/index. Happy Analysing!
For once i thought i'd do an off topic post that doesn't have anything to do with Mobiles, HTC or ROM's. About 6 or so years ago, i joined a project online called 'Majestic 12' which purpose is to build a search engine using distributed computing. Being into the whole 'distributed computing' thing at that time i decided to join up and help contribute a few thousand urls or two using it's MJ12Node program. A few months of crawling and i'd already crawled an amazing 500 million urls! This was amazing considering that 1 PC in 6 months on the internet, what could 1000's of PC's do?
I stayed with the project for a few more months, helping with suggestions with the node and helping make a help manual at the time to help explain all of the buttons, options and features in the software. As life tends to do, i moved on and completely forgot about Majestic 12, it's staff and distributed computing as a whole.
Forward to a few days ago, i received an e-mail off the founder of Majestic 12 with the following message:
Hi, You are receiving this one off message because you took part in Majestic 12 Distributed Search Project and earned entitlement to a share of profits that were allocated in December 2010. You have 3 options: 1) ignore this email and forfeit your share of money 2) provide your payment details (Paypal only supported) to get your share 3) provide only your project details and choose your share to be given away to charity instead If you wish to submit your details then please read this forum thread: http://www.majestic12.co.uk/forum/viewtopic.php?t=4557 For your convinience we've created special thread that details history of the project over last 7 years: http://www.majestic12.co.uk/forum/viewtopic.php?t=4653 You have 2 weeks until 25 Sep 2011 to submit your details. Regardless of your choice I'd like to thank you for your contribution to the project and apologise for having to email you this message - we will NOT do so in the future unless you submit your payment details. Best regards, Alex Chudnovsky Managing Director Email : firstname.lastname@example.org Web : http://www.majestic12.co.uk Majestic-12 Ltd (t/a Majestic-SEO) Faraday Wharf, Holt Street Birmingham Science Park, Aston Birmingham, B7 4BB
It's amazing to think that after such a long time i'd be reminded of my work towards Majestic 12 and that i was owed some money for my efforts. So i decided to start crawling again and help out in anyway that i can.
So you're probably thinking, what does this mean to me reading this post on the internet. It means that we all have a part to play on the internet and we can all help build something out of nothing. If you want to help this project, i recommend it. You can join up at http://www.majestic12.co.uk and download the node and help crawl a few urls to help build the search engine index.
Also for your SEO needs, you can find out more information about your own site at: https://www.majesticseo.com/