Commodity: (from Wikipedia): A commodity is a good for which there is demand, but which is supplied without qualitative differentiation across a market. It is fungible, i.e. the same no matter who produces it. Examples are petroleum, notebook paper, milk or copper.
Would you classify; hacking, security testing, targeted vulnerability analysis and research, etc – activities that in one form or another come under the banner of “penetration testing”, as a commodity? Many do…..wrongly!
It seems to be a pattern that the larger the consulting organisation, the greater the drive to rapidly “commoditise” those activities that are; not core to the business, stress* resource capabilities and have less profit margin, (but are a necessary part of their business to compete). The end result is generally an attempt to outsource these capabilities to cheaper labour to relieve the “stress” and to increase profit margins. (*“stress”, in the above scenario: issues, pressures and costs associated with attaining and maintaining exceptional quality people).
Is the assumption, that with a little bit of training and the right tools, anyone can deliver this [penetration testing] work, insulting to the people who are experts in this field? Of course it is. (Even outside the context of “commoditisation”, the topic at hand – you can argue validity on skillset alone for individuals and/or organisations, who don’t view it as a commodity service, but rather market themselves as experts when they are not).
I can see an argument for the commoditisation of penetration testing – but only in a world where nothing is changing, tools mature to cover most likelihoods and scenarios, and a general awareness/expertise level where such knowledge is no longer the differentiator it once was. This is not the world we live in.
Historically we have learned that “outsourcing” can have a detrimental impact upon quality of service, reduced ownership/awareness/oversight/visibility…and security. Valid points in this discussion in my opinion.
The other day I read somewhere someone promoting; “Penetration Testing from the Cloud”? WTF is that? If a client of mine is rolling out a new technology – hardware, software or both, is some outsourcing mob going to be able to effectively test the security of this new system for my client? I doubt it! For businesses dealing with organisations that have self-determined that penetration and other security testing can be done by sweatshops, will they know that their business is being serviced by such sweatshops, (fronted by a reputable name)?
I acknowledge you can commoditise certain things – well to a degree at least…..and even then, you still have to have the caveats in place. As an industry, we are still young and struggling to get even the basics/fundamentals across of Information Security to the broader community. Commoditisation in most cases for our industry is detrimental to the cause. Taking the intelligence out of things is just plain stupidity. Realising it [commoditisation] is being done in most cases to increase the profitability of a company whose focus is purely to make money from you should make you question and thoroughly assess what it is you are buying and whether it really is providing benefits to you.
You can’t run an F1 car on dirty 91 RON. (And if you want to argue that your business is not an F1 car, but rather a Toyota Camry, ask the owner or CEO if he agrees). 
———————————————————————————————-
Securus Global: IT Security, Penetration Testing, Security Assessments, PCI Compliance, Product Assurance, QualysGuard, Security Strategy, Vulnerability Assessment.
It’s important to stress that the methodology can be trained. A willingness to learn, technolust and a desire to pwn simply can’t be. Commoditisation means we are seeing more formal approachs in the market for teaching approaches to pentesting, even _some_ of the technical skills required. But the depth of technical talent will always show up eventually. That isn’t something that can ever be fully taught or commoditised.
It’s one thing to have an amateur follow a checklist.
It’s another thing to have a seasoned professional follow his checklist but know when to go off the beaten track and follow an instinct.
That is the difference between good and great.
Whats interesting is I’ve seen at least half the companies trashed in this thread and I can honestly say some are much more consistent performers than others. I’ve seen some companies sail by on technical skill alone, but others who may not be as technically gifted do ‘better’ simply because of their repeatability.
Before some people start trashing other companies here, you should actually take a good long hard look at their templates, reporting, methodology and approaches. You can actually learn a lot from them. Take the bits which you could learn from, scrap the bits you don’t like… and laugh at the stupid findings when the crop up!
No offence to any individuals whose comments have been removed. I wanted the focus to be on the topic and not individual organisations.
You could hire these guys:
http://www.ligattsecurity.com/
Free tips on Twitter also making it easy for all!
http://twitter.com/ligatt
Many of these bigger consultants should not be in this business. Why delete the truth? Ever seen an EY pen test report?
Ok. I’m not bashing on anyone here but umm, that LIGATT site – I can’t tell if its for real or a joke. I think its real however and that makes me sad.
The delivery is based upon the quality of the people. A checklist, no matter how good will not turn someone into a specialist. Therefore you must always question quality of delivery to a customer.
Some previous names mentioned here developed a bad reputation deservedly. Clients were ripped off and some continue to be. Why defend them? Most of them are here when they smell the money and then leave when it’s dry. They do not have any attachment to the infosec industry.
It makes business sense for them to dumb down pen testing to bring it to a level they can deliver. They can’t compete with Securus Global, Sift so disrupt the market to the detriment of the bigger cause.
Basic VA is commoditized and this is what they do. I can’t see experts and researchers to the calibre of what we have in Australia being out of a job for a long time. Let the big consultants go for it. It will just widen the gap between the quality firms and themselves.
[...] Commoditizing Penetration Testing? – I really like how this post defines what a commodity is: something that is the same no matter where it is produced, like paper. You don’t want your pen test to be like paper; penetration testing is more dynamic and custom. If you commoditize it, you may end up with a report that is about as useful as a blank piece of paper. [...]
I don’t have a problem with VAs in a general sense but the problems I see is not the VA itself but how they are often sold (at a commercial level) and not enough time attached to the back of them to explain the findings to the customer, difference between a VA and a pentest and spend more time focusing on the remediation effort than scaring the shit out of them with a sea of findings.
To be fair, I have seen some of those companies bashed for selling VAs and having seen the manner in which its been done, I can say yes it is deservedly. However, they are not a pentest so lets make sure we’re comparing apples with apples.
There is also good guys in Big 4. Saying that E&Y consultants are bad is as retarded as saying that StratseX consultants are good :/ It mostly depends on who you get to do your pentests.
If i was a CSO, i will ask for names on proposals and check who is doing my pentests…
This is correct Blah but it is the pattern over time also.
Go see who is number 1 according to the authority, Google! You other noobs are just pretending!
Does StratSec do penetration testing?
@”Sense it Smell it…that’s L33T” i think they ./consult
If only there was an Australian certification body to iron out the noobs and pretenders then we would not need to have these conversations.
@Sense it Smell it… etc
The “if only there was an Australian certification body” theory is fine, but it obviously won’t be free.
So let’s say it costs $5-10K to do the necessary “validation” tasks for skills etc to be a tester. All we do is replace our “false positive” problem (ie people claiming to be skilled who aren’t), with a “false negative” problem (ie people who are skilled, but can’t afford the fees)…
Yes Stratsec do pentesting. Yes they do a good job. With Blah -> ask for names. We get that as a client request (particularly for big gigs all the time).
Jarrod Loidl responds with some good points:
http://jarrodloidl.blogspot.com/2010/06/counterpoint-commoditising-penetration.html
who is driving the “commoditising” of the service, the consulting organisations or the market? I don’t think it’s as black and white as the author has made it out to be.
If a new client approached Securus Global and said we have 1,000,000 apps to do over 5 years and we only want to get a general feel of the security of the apps and as such repeatable testing is more important than a complete in-depth pen test would Securis walk away based on the fact that they will be forced to commoditise the service? I think not.
Hi Jimbo,
You are right. It is not black and white and one could argue that even the likes of PCI is “commoditising” pen testing.
We support a position that we believe in but there are times where we are asked to work outside of what we believe is “best”.
In the scenario you describe, you’re also right. Of course we would not and we’ve worked these scenarios many times and continue to do so. In every such circumstance, we strongly highlight the limitations of what such “commoditised” / automated testing are – both in pre-engagement and in the reports we deliver. The client needs to and must understand exactly what they get as I have no doubt you’d agree.
In all circumstances though we try to, best efforts and within timeframes we have, and through experience determine where further investigation is really a must. (Above the limitations we’ve already highlighted). Can those delivering just “commodistised” testing do that? ie; experience and gut instinct see in reports some clear results that can/may/will lead to discovery of a larger vuln not reported by a scanner? etc etc….you get my drift?
SG doesn’t count the hours so to speak so we really try our best to deliver the best result we can within the limitations we are asked to work within. Not a marketing statement here but a principle we work with.
What’s your position Jimbo?
DD
—————————-
Tenable Podcast: (just an FYI….no more and no direct relation to this post)
http://blog.tenablesecurity.com/2010/06/welcome-to-the-tenable-network-security-podcast—episode-37.html
agree with everything in the reply. Point being though is that most people do it but are not nessarily behind the idea/scope.
My position is that it’s a combination of tool vendors and client knowledge (or lack thereof) that are behind the drive to commoditise pen testing. i.e network world and Nessus.
In my experience it’s not the larger consulting organisations that are behind it, I can’t see them actively saying “no need to pay us top dollar as pen testing is easy” they would be more inclined to simply put an inexperienced tester on the job. Some have a good QA process behind that and some don’t.
In my opinion, the real cancers in our industry are the smaller IT firms out there (not nessesarily 100% security) that sell automated tool output as legitimate web application tests. Takes them an hour or two to run the scan and can sell the service for $20. I’ve seen reports where the tool was obviously not even configured to log into the application. I look at the client, the client looks at me and says “see… we’ve done this and we’re secure”
Could just come down to different experiences.
Ask Deloitte if they have a team in India positioned to deliver this commodity?
A commodity is a product that is completely undifferentiated.
Having the power to deliver it for less is not commoditising.
Jimbo. You must be a director or partner to produce that. Your team must feel so appreciated. For your clients, that definition is your expectation. And bigger profit margin the goal. Go ask the horses. They tell everyone else and you as an insider don’t know? Or just trying to convince people here? Rofl. Good luck.
As Jim Quigley I take offence that you called me a mere Director or Partner. lmaorofllolasap
Im out of here i stumbled across this blog post after recently having a similar debate with collegues and didnt mean to open up a can of worms as they say
nice blog though
Jimbo,
Great to have you here. Nothing better than a bit of debate. Awesome you take the time out of your busy schedule running Deloitte to join us here.
Have a good weekend.
DD
Consultancies are only going to offer the services that have been requested by their clients. Clients are only going to request services they think they need. Few consultancies really challenge their clients over their assumptions, ask tough questions to distill their actual requirements, etc.
Both sides are to blame for this state and it will take both to change it.
Lets be clear commoditising is not a bad thing. Providing a shitty service is.
Jay,
Your comments are generally on the ball but I need to pull you up here.
I agree with your first comment(s). But….it does come down to differentiation.
When we first started SA and then SG, the market circa 2001/2002 was full of ordinary delivery from big consultancies.
We knew it should be batter and what it took to make it better. We did that! Few now acknowledge what SA/SG and companies like us (SIFT and others) did. We changed the market here in Australia and NZ. The Big companies had to follow. That is why companies like us were able to grow and turn some of these “consultancies” into reactionary bodies and why some Big companies no longer can compete in this market!
DD
I meant I needed to argue a couple of points Jay…..not the whole thing as my opening line can be taken to mean.
The difference between a basic test and a good test is gigantic. Getting juniors to run tools is not pen testing.
I wasn’t involved back in 2001/2002 really so I can’t comment on that.
On the subject of pentesting, I think Securus Global really do step it up a notch in terms of identifying technical risk. Anyone that’s hired SG and other firms I think would agree with me.
I was just generalising my comments that some people think that a lot of people providing pentesting is a bad thing. I dispute that and my point is only that more pentesting isn’t bad – providing a crap service however will always be bad to the consumer (whether its a pentest or going through the Maccers drivethrough).
Speaking of crap service, can anyone explain WHY that regardless of what Maccers drive through I go through, no matter whether I order six items or two, they ALWAYS find a way to screw it up? I mean ALWAYS. WTF??
- J.