Ask any human resources manager how they determine employee salaries and you’ll soon find out just how complex salary data has become in recent years.
In the past, HR consulting companies gathered data from a broad cross section of employers and then sold the reports to human resources managers to help them make informed decisions. Employers that could afford it had access to a handful of expensive but reliable sources for compensation-related data; employees had to make do with the federal government’s annual Bureau of Labor Statistics reports and whispers around the water cooler.
Today, thanks to the sharing economy, a multitude of crowdsourced salary data websites collect data by requesting anonymous information from site visitors about salaries, bonuses, and company reviews. Sites like Glassdoor.com, PayScale.com, and Salary.com all report on millions of anonymous user contributions and aggregate that data with everything from geography and industry, to job title and corporate function.
To the untrained eye, these crowdsourced salary data sites offer an embarrassment of riches for human resources managers -- there is data as far as the eye can see, and it’s not restricted to the folks that can write big checks. However, employers who dig a little deeper into data volume and availability will soon find out that crowdsourced salary data doesn’t always live up to the hype. Serious questions remain regarding the accuracy of the data; the best way to use it; and the best way to adjust your rates when the data doesn’t quite match up with internal practices.
There are two critical drawbacks all employers should understand when considering the pros and cons of using crowdsourced salary data:
Crowdsourced Salary Data Isn’t Consistently Cataloged
Consider how frustrating it is when you are collaborating on a filing project and a colleague uses one name for a file, and you use another name. Even though you’re working on the same project, it’s impossible to achieve your goal because you’re using different names for the same thing. And that’s exactly what happens when crowdsourced salary data websites catalog data at mass scale without using consistent or universal job titles or organizational roles.
Consider the example in Table 1 below, which compares three independent sources: GlassDoor.com; the Bureau of Labor Statistics (BLS); and the HRA-NCA Compensation Survey, an annual survey conducted by an HR professional organization in Washington DC.
Please note that this data is representative of what is available on the top crowdsourced salary data platforms. The data presented is not meant to single out any sources as either more or less accurate, more or less representative, higher or lower quality, or any other such qualitative measure.
The differences in labeling alone can lead to significant confusion and often an inability to draw meaningful conclusions. The first job title, Software Developer, Applications, exists in all three surveys in very different ways. GlassDoor.com reports the Average Salary while BLS and HRA-NCA both report the Median Salary. Comparisons between GlassDoor.com and any other survey are therefore difficult, if not impossible.
BLS and HRA-NCA have significant differences as well. HRA-NCA breaks out the Software Developer, Applications, into five different levels, while BLS does not.
The second job title, Software Engineer, only exists in GlassDoor.com. One could speculate that the Software Developer, Applications (Level III) from HRA-NCA, the Software Developer, Applications, from BLS, and the Software Engineer from GlassDoor.com are probably referring to very similar jobs. But that’s all it would be: speculation.
Method and Sample Size Heavily Impact Accuracy
Most statistical methodologies begin with a simple premise of random sampling, or choosing samples at random to make it equally likely that any particular individual item will be selected as part of the sample. When a using a random sampling, statisticians apply certain adjustments to account for errors because it’s generally accepted that some level of error will exist. And this is just where crowdsourced salary data breaks down in reliability:
Not only are crowdsourced salary data websites not working with a random sampling of data, but they often reach conclusions based on very small sample sizes. For example, GlassDoor.com, PayScale.com, and Salary.com’s sample sizes are quite small relative to the entire population. Comparatively, BLS offers the best overall sample size while HRA-NCA stratifies the data according to levels supplied by its survey respondents.
The Best Alternative to Crowdsourced Salary Data Sources
Considering the average company spends forty to eighty percent of its gross revenue on salaries and benefits, it’s more important than ever to ask whether or not crowdsourced salary data holds true statistical value. After all, aggregating errors of only one or even one-half percent can result in significant errors in budget estimates. And those same errors, when taken individually, might result in the loss of a key contributor to a competitor when an employee finds significant differences between what they make and what they’ve found on a crowdsourced salary data site.
Depending on the sector of your business, you may spend between 40 to 80 percent of gross revenues on employee salaries and benefits combined. Salaries alone can account for 18 to 52 percent of your operating budget.
-Society for Human Resources Managers
Instead of attempting to navigate these disparate data sources, the most strategic and stable approach to setting employee pay rates is to secure a high-quality salary survey from a reputable source. While there is an associated cost with most surveys, they vary dramatically according to your market, so you can often secure a survey at your budget if you shop around.
To get started, look for recommendations from local chapters of professional human resource associations, like the Society for Human Resource Management (SHRM) and WorldatWork. Then consider the following elements when interviewing a survey partner:
Having identified your base survey(s), selectively compare what salary ranges they indicate for jobs that have been deemed critical by senior management. The rationale for this is twofold:
Which leaves us with a much clearer picture of the purpose of crowdsourced salary data: it’s valuable information to help you compare and refine your decision-making process, but it’s simply not statistically relevant enough to single-handedly support your salary decisions. The more informed and prepared you are, the better decisions you can make, and high-quality salary surveys from a reputable source are still the best informants in the industry.
About the authors: This article, courtesy of Al DuPree, COO of AKRON, Inc, Jordan Hayes, Research Associate, and Jennifer Barbee, VP/Marketing. AKRON, Inc. is an HR survey and data analytics company located in Washington, DC. AKRON is a leader in providing substantiated, empirical data to empower HR departments in their plight for organizational excellence and talent retention. Since 2004, AKRON has been the administrator of the HRA-NCA Compensation and Benefits surveys.