| LED Digest 2245: Improving Conversion Rates |
|
|
|
A fantastic post today from Sumantra Roy on Taguchi multivariate testing. Sumantra says that it is virtually impossible NOT to get a significant improvement in sales after every test. Also, we revisit The Death of Email Publishing thread which ran in early 2005. ================================================== The LED Digest Moderated Discussion List "Effective Online Advertising, Since 1997" Data > Information > Knowledge > Wisdom pair Networks: The LED's Web Host Hosting and Domain Registration from a Trusted Leader pair.com for Hosting | pairNIC.com for Domains ================================================== List Moderator: Published by: Adam Audette LED Digest adam, led-digest.com http://www.led-digest.com .............................................. September 13, 2006 Issue no. 2245 .............................................. .....IN THIS DIGEST..... ====== NEW ====================== <Moderator Comment> ~ Revisiting the Death of Email ==== CONTINUING ================= --== Taguchi Multivariate Testing ==-- ~ Nick Usborne "...I think I can serve you best by providing links to two of our research briefs." ~ Sumantra Roy "...it is virtually impossible NOT to get a significant improvement in sales after every test." --== Small Business Sites ==-- ~ Jim Gatton "I'm not sure we ever even qualified as a small business...It was just survival." --== The Search Guru ==-- ~ Dirk Johnson "ROI discussions of the various options are almost unheard of within SEO." ========== NEW =================================== <Moderator Comment> Greetings LEDer, I thought it would be interesting to revisit the thread on "The Death of Email Publishing" which ran during the early part of 2005. The discussion evolved from threads on spam problems, email publishing alternatives, and evolving technologies such as RSS. In a nutshell, this topic asked, "how will email publishing evolve in light of problems with email and more efficient emerging technologies?" ------------------ Background reading: LED Digest 1937: Email Publishing Alternatives http://www.led-digest.com/content/view/971/55/ LED Digest 1938: More on Email Delivery Alternatives http://www.led-digest.com/content/view/972/55/ (the above issues can be accessed from this index: http://www.led-digest.com/content/category/5/29/55/ ) ------------------ With the benefit of time it would be interesting to discuss this again. I can start things off. After implementing RSS feeds on the led-digest.com site I've seen a steady rise in feed subscribers over the past 6 weeks, but an even steeper spike in traditional email subscribers. It may have something to do with how I'm presenting the RSS feeds - basically in headline links to the full version online. I wasn't expecting anything major, and I guess I'm not surprised. I still prefer email, and with good filtering don't have much of a problem with spam. Just some thoughts off the top of my head. I'd love to hear yours. Email publishers: have you implemented alternative methods for delivering your content? How is RSS helping you connect with a larger readership, and how is it being combined with other aspects of content management, such as blogs and forums? Looking forward to the response, Adam ======== CONTINUING =============================== From: Nick Usborne Subject: Taguchi Multivariable Testing > We have been thinking of using Taguchi multivariate > testing on our web site in order to improve its conversion > rate. Does anyone have any experience with Taguchi testing...? - Valerie Walker, LED 2244 Valerie, those are some great questions. I could write yards on this topic, but I think I can serve you best by providing links to two of our research briefs. They're quite long, but cover the ground thoroughly with plenty of test data. A/B Split testing http://www.marketingexperiments.com/improving... Multivariable Testing http://www.marketingexperiments.com/improving... I hope this helps, Nick Usborne, Senior Editor www.marketingexperiments.com -------- new post - same topic -------- From: Sumantra Roy Subject: Taguchi testing Valerie, We've had excellent results when we have applied our Taguchi multivariate testing methodology and software on our own sites and our clients' sites - we've been able to improve conversion rates and revenues by 37% - 85% after only 1 round of testing. However, there are a few things you should be aware of before you get into this: 1) Taguchi testing is based on using some mathematical arrays (called "orthogonal arrays"). However, the problem is that MOST of the Taguchi arrays that are present in standard text books in Statistics do not work too well when it comes to improving web site conversion rates. So, only a few of those arrays should be used for this purpose. When we were developing our own multivariate testing software and solution, we had to put in a HUGE amount of research (and had to literally spend more than a $100,000 in consultant fees, traveling costs, the value of my own time etc.) to figure out which of those arrays work and which don't work too well. 2) One of the problems with Taguchi testing is the fact that it does not do a very good job of taking into consideration the fact that the factors that affect your web site's conversion rate (by "factors", I mean things like you Headlines, Prices, Navigation Structure, Copy, Layout, Images, Guarantees, Bonuses etc.) INTERACT with each other. When I say that factors "interact" with each other, I mean that the impact of one factor on the conversion rate depends not only on that factor itself, but also on the OTHER factors that are being tested. Here's an example: Price and Bonus are 2 factors that have a strong interaction between each other. This is because the impact of Price on your conversion rate depends not only on the specific price being tested, but also on the bonus being tested. Your customers would be willing to pay a higher price for your product if the perceived value of the bonus that you are offering is higher. Therefore, while a higher price would normally be expected to reduce your conversion rate, it may not do so IF you also offer a much more valuable bonus than what you were offering earlier. Other factors that interact with each other include your Price and Guarantee, your Headline and Sub-Headline etc. (there are lots of such examples). The standard Taguchi formula that is used to analyze the results of a multivariate test assumes that factors do not interact with each other. Because of this, they are not able to generate as much of an improvement in the conversion rate as could otherwise have been done. Because of this, we had to invent our own modifications to the standard Taguchi mathematical formula in order to take into account the interactions between the factors. 3) Lastly, what we have found is that, apart from the 2 mathematical problems I mentioned above, one of the main reasons why people often get less than stellar results after using multivariate testing is that they don't know WHAT to test. The mathematical formulas all answer the question of HOW to test, but in order to really get the most from multivariate testing, you need to know WHAT FACTORS will tend to have the highest impact on your conversion rates, and then know the different variations ("Values") of those factor that you should be testing. And this is where most people who are new to testing go wrong. For instance, one of the common mistakes that people who are new to this make is to test a factor like their Font Type. What we have found after several tests is that whether or not you have used Arial or Verdana or Times New Roman or Helvetica or whatever has almost NO impact on your conversions. I am yet to see a single split test or multivariate test where the Font Type made a very large difference in the conversions. It's like the 80 - 20 rule that all of us are familiar with - 20% of the factors will account for 80% of the improvements in the conversion rate. Therefore, in order to get the most out of multivariate testing, you need to know which factors are among the top 20% and which are not. If you don't have too much expertise with testing, you are better off hiring an expert to implement the test for you, rather than trying to do it yourself. For instance, in our own company, when we were developing our multivariate testing methodology, we first amassed a huge knowledge base of factors that can be tested on a web site and then, after conducting hundreds of tests, identified which specific factors in our knowledge base have tended to produce the largest improvements in the conversion rate for our own sites and our clients' sites. We now concentrate on only those factors when we run multivariate tests for our clients' sites. You also asked whether there were any differences between Taguchi testing and A/B split testing, apart from being able to test multiple factors at the same time. I'd say there's 1 more: In far too many A/B split tests, the control version of your site out-performs the challenger, which means that after all the testing time you've invested in conducting the test, you won't have any sales improvements to show for it. You do learn something from each test, even if it fails, but I'd much rather learn something AND get some sales improvements at the same time :-) With multivariate testing, you can test so many factors at the same time that it is virtually impossible NOT to get a significant improvement in sales after every test. Hope that helps. Sumantra Roy http://conversionmultiplier.com/taguchi-multivariate-testing.htm white paper on Taguchi multivariate testing. -------- new post - new topic -------- From: Jim Gatton Subject: Small Biz Websites + More time than money > I am fully confident in my skills doing these things > I mention, but with more money than time I have hired > pros to do the work at times... When you are already > a success and have the luxury of paying a pro, it is > easy to do so. - Brad Waller, LED 2244 Wow, can I ever relate to that comment. I was a tiny part of an online discussion recently where a merchant indicated that it was going to have to make some changes in how they do business with us. Not a reduction in commission or anything horrible like that. Just some changes. Many were whining about it. It was just a change. A few acted as if they were in agony. Hey, an otherwise rotten District Manager taught me 40 years ago about being alert to "shifting priorities" and doing "whatever it takes" to get the job done. Just learn the new skill, make the change and move on! Now let's discuss operations that don't even qualify as small businesses and see how it might fit into this context. My wife and I ran a little momandpop inner city corner grocery store for ten years about thirty years ago. I did the basic bookkeeping, swept the floors, worked all day, etc., One time after all of that we closed the store, removed the fixtures and tiled the floor, replaced all of the fixtures and re-opened shortly for business. There were a lot of times like that. We didn't have any choice. But, we did survive. It was only because we were willing to learn how to do a lot of things that we survived. It was the first floor I had ever tiled but we had more time than money. While I'm willing to work hard long hours I don't have that intangible edge that translates to success. Success would have meant that I could at least pay others to do some of the many tasks on my long list and, -TA! DA! - become even more successful. No, we were just trying to survive. I'm not sure we ever even qualified as a small business except on some kind of technicality. It was just survival. We skipped lots of sleep to get the time because, you're right, Brad, we had more time than money. That was in the 70's and nothing has changed. Some lives are just like that, friends. I have more time than money so I have to learn to do a lot of things. It's thirty years later and I'm still in survival mode but seeing opportunity in the internet. You see, people don't hire 60 year olds without marketable skills / education. So I learn SEO and SEM and a little bit of web design, etc. Before Google and the others (justifiably) broke my arm (forget slapping my wrist) I was #1 on Google, MSN and Yahoo for some extremely competitive insurance terms a year or so ago. It didn't really relate to making much money though. Didn't last long enough, in part. So, I had to keep learning and doing it myself. I still don't qualify as a small business, I'm in survival mode, and I'm surviving. I'm surving in part because I've learned to do a little bit of everything in this business. As a result of adapting and learning I have food on the table, a roof over my head, gas and electric service in my house. I don't have the luxury of paying a pro. Sure, I'd love to talk with someone that could drive quality traffic / loan leads to one of my sites. Pay for performance kind of thing. But, I really do not have the luxury of committing large sums of money on other than an "as earned" basis. Good luck with that! So, I keep doing these little things myself. I get by. I suspect there are a lot of others in the same situation. So, I'd agree that for any business with some realistic cash flow it's much better to invest with the professionals to get it done both quickly and well. However, there are many in the world that literally don't have the cash flow to invest in professionals so we learn to do a lot of different things. I work on my own car, do my own basic electrical wiring in the house, do cement work for my friends, PPC in the office, web design on the laptop. Sometimes we just don't have the luxury of paying a pro but we can learn to do a lot of things and do them reasonably well because we have more time than money. So, thanks to everyone that shares the tidbits here that help us smaller-than-small-business types learn how to survive. You're appreciated. Jim Gatton http://loans-finder.org.uk/ -------- new post - new topic -------- From: Dirk Johnson Subject: Search guru > Why are people apparently up in arms when > recognized and established names in a very > well established industry speak regularly at > conferences or post knowledgeable advice > and commentary? - Mike Banks Valentine, LED Digest 2243 Certainly, when it comes to linking, I can find example after example of so-called "recognized and established names" making some kind of public comment that simply falls apart under close scrutiny. Quite a few of the best known SEO advisors have published statements about linking that, at times, border on the ridiculous. To me, they readily reveal their own lack of understanding of certain aspects of linking and how it affects the search engines. What's my basis for saying this? Actual search results. When there exists pervasive evidence that contradicts what these "experts" claim, then it becomes a matter of credibility. Even when presented with such evidence, the pundits often still cling to their myths, theories, and other groundless concepts. So it's become less of a search for truth than a means of selling a particular SEO-related solution. From what I've seen, most SEO houses have a favored approach, especially toward linking. They do not advise their clients as to the whole scope of what is possible, the various costs, the expected return on investment, and other aspects of the world of linking. Instead, they advise the solution that they are selling, and they will defend it against all others, as the only "right" way. Just to clarify, while we sell only reciprocal linking services here, we always advise our clients to pursue as many legitimate methods of link development that they afford and manage. We're just one piece of the puzzle, and we are quite comfortable admitting to that. We tell them to leave no stone unturned, if their budget can handle it. Some clients are more determined or more capitalized than others, which provides them with more options. But all possibilities have to be evaluated as to the expected return on investment, with the most advantageous methods pursued first. Every situation is different. In a lot of cases, the best ROI may not be with reciprocation, or reciprocation may not be practical for some sites. We regularly turn away clients for whom we do not feel confident that what we can help in an affordable, timely way. Frankly, I don't often hear that kind of ROI comparison coming from many other places in the SEO world. In fact, ROI discussions of the various options are almost unheard of within SEO. I find that to be very curious. Instead, I hear a lot of scare tactics, with the implication that pursuing other tactics than the one currently being sold is dangerous or ineffective. I see claims of "white hat" methods that, when reviewed, turn out to be not "white hat" whatsoever, and are instead just easy gaming tactics. They can be expensive, sometimes even high risk, but they are easy to manage and easy to sell for the SEO firm. That's not advising the client with their best interest in mind. An SEO should be able to evaluate ALL the possibilities, and put forward a customized solution that fits the client's situation and budget. One-size-fits all solutions can work, but that's not really SEO consulting. That's just selling a service. SEO is hardly a "very well established industry". As others here have stated, there are no standards whatsoever. Anyone can claim to be an SEO expert, and then begin a program of effective self promotion. A couple of years ago, I even recall an article that was well-referenced in SEO circles that focused on how to make yourself an SEO guru. The thrust of it was to just start writing, posting in forums, attending events, making friends of insiders, etc. There is an awful lot of "who you know" in this industry. Skill, understanding, and real experience in this industry derive from actually doing the work (a lot of it, daily), with eyes wide open, and no pre-conceived bias. Quite frankly, too many people now confuse "celebrity" and self-promotion for skill, understanding, and experience in this industry. Fresh voices with unique perspectives are very hard to find. You have to examine carefully what is being said, and why, and not let the presentation forum sway your opinion. Yes, there are SEO consultants that know their stuff, and yes, some of them are speakers at the conferences or are well-known writers. I know a some of them personally, and we have working relationships. But they are more interested in providing services to clients than in making a name for themselves. What's more, some of the most skilled SEO "gurus" that I know are individuals that run private websites, and they are not for hire. I have had numerous discussions with these people about the state of advice coming from some of the big names in the SEO industry. Like me, they've learned to ignore most of it. When you remove the need to need these SEO celebrities for advice, it's liberating. The celebrity status of an SEO consultant means very little to me. People can hire big name SEO consultants, or follow their various prescriptions for success, while feeling that the advice has been vetted by others. But depending upon the particular advice, it can be very expensive. I'd bet that they are frustrated when a skilled, knowledgeable competitor uses more mundane, proven, affordable grass roots web promotion methods to accomplish even better results, for a lot less cost. I see that kind of thing take place all the time in this business, so, in my mind, it all washes out. Good SEO practice is remarkably inexpensive, once you know what to do. There is a huge premium that is paid by the unwary. Again, it comes back to return on investment. A fortune can easily be spent on misguided, limited-scope advice, but eventually it needs to be accounted for, in a competitive environment. Unfortunately, for the uninformed who feel the need to follow such bad public advice, under the assumption that "well-known" equates to "skilled", it can be costly, in both time and money. Its not about who does the work, or spending large sums. It's about doing the most effective things, and not allowing bias to drive decisions. As I said, it all washes out. Best regards, Dirk Johnson, Partner - Operations DomainDrivers LLC www.domaindrivers.com www.linkstrategy.com ------------------------------------------------------- The LED Digest is sponsored by pair Networks: pair.com for Hosting | pairNIC.com for Domains Copyright 1995-2006 Orange Wheel, LLC. All Rights Reserved. ----------------------------------------------------------------- "In a cat's eye, all things belong to cats." - English Proverb |
