Group composition

S.No	Name of Student	Email
01	Muhammad Imran	Muhammad.imran91@stu-mail.sps.cuny.edu
02	Kenny Jeannite	kenny.jeannite72@cuny.edu
03	Samantha Mesa	samantha.mesa84@login.cuny.edu
04	Tenzin Thakuri	tenzin.thakuri35@login.cuny.edu
05	Shanice Smith	shanice.smith16@cuny.edu

1. Introduction and Background

After an in-depth discussion of the research question, “Which data science skills are most valued?”, the group concluded that addressing this topic requires an empirical, data-driven approach that goes beyond merely listing ubiquitous programming languages or routine, repetitive tasks. To establish a rigorous foundation for this study, it is essential to first define what constitutes “value.” In this regard, the group asserts that the value of a data science skill is best measured by its ability to create distinct differentiation within an organization.

Rather than viewing skills through the lens of basic technical proficiency, true value manifests when competencies enable a data science team or individual to drive competitive advantage, solve non-standard business challenges, and propel strategic transformation. Under this framework, organizational differentiation can be operationalized across three key dimensions:

Strategic Impact vs. Commodity Work: Differentiating skills move professionals away from replaceable execution (e.g., standard baseline modeling or basic reporting) toward high-leverage capabilities, such as translating complex domain problems into actionable algorithmic frameworks or designing end-to-end architectures.
Problem-Solving Complexity & Novelty: Value is defined by the capacity to tackle unstructured, ambiguous problems that off-the-shelf automated AI tools cannot resolve. Skills in advanced statistical inference, causal modeling, and custom system design allow organizations to innovate rather than merely keep pace.
Value Creation and Measurable ROI: Skills create high differentiation when they directly influence enterprise key performance indicators (KPIs)—whether by optimizing core operational workflows, unlocking new revenue streams, or providing proprietary insights competitors cannot easily replicate.

By defining value through organizational differentiation, this work will try to establish a quantitative framework to evaluate market data, job market premiums, and business outcomes, ultimately pinpointing the exact capabilities that elevate data science from a functional utility to a strategic asset.

2. Proposed Analytical Pipeline

Step A: Environment Setup & Data Ingestion

This step loads the required libraries (tidyverse for data manipulation and ggplot2 for visualization) and imports your dataset from the github where I stored the data files, with robust character encoding. After the loading process the following table of information loaded in to the dataset;

Dataset Component	Total Count
Rows	1036
Columns	40

Step B: Data Cleaning & Outlier Management

Survey datasets frequently include extreme compensation outliers, zero values, or missing financial records. This step cleans CompTotal by removing non-positive entries and filtering out extreme outliers above the 99th percentile.

The code snippet under this step generates a clean, formatted presentation table that summarizes the physical size of your dataset (df). Specifically, it performs the following tasks:

Loads the knitr package: Imports the knitr library, which is built into R Markdown and Quarto environments to render data frames into professional Markdown tables.
Extracts and structures dataset dimensions: Uses nrow(df) to count the total rows and ncol(df) to count the total columns, organizing these counts into a small, two-row data frame called dimension_summary.
Renders the final table: Uses the kable() function to convert the summary data frame into a styled table with custom headers (“Dataset Component” and “Total Count”) for your final output document.

## Cleaned dataset records for financial analysis: 750

## Median Compensation: $105,000

## Mean Compensation: $565,998.6

Step C: Text Parsing & Skill Extraction (tidyverse Unnesting)

Survey columns storing multi-selected technologies (e.g., LanguageHaveWorkedWith) use semicolon separators. Using separate_rows() tokenizes these strings into individual skill observations.

The chart below highlights the most widely utilized technologies based on survey respondents’ work experience. JavaScript leads the rankings with 501 respondents, closely followed by HTML/CSS (483), SQL (460), and Python (417). Mid-tier entries include Bash/Shell (352) and TypeScript (327), while Java (216), C# (214), PowerShell (189), and C++ (159) round out the remainder of the top ten.

Step D: Economic Differentiation Analysis (Compensation Premiums)

To substantiate your research argument that true skill value is driven by organizational differentiation (rather than routine execution), this step groups respondents by developer roles and advanced skill sets, calculating median and mean compensation premiums.

The empirical data demonstrates that specialization and market differentiation serve as the primary drivers of earning potential in the tech industry. While generalist roles such as full-stack developers represent the largest workforce segment ($n = 230$), they command a baseline median compensation of $100,000.

conversely, specialized positions like DevOps engineers ($n = 17$, median $144,700$) and data engineers ($n = 18$, median $137,500) yield premiums upwards of 37% to 44% above generalists due to talent scarcity in complex infrastructure domains.

A parallel trend emerges in technical skill sets: high-adoption, foundational tools like SQL ($n = 460$, median $112,500$) and Python ($n = 417$, median $113,000$), show lower median earnings compared to niche, high-demand languages like Go ($n = 117$, median $138,000$) and Ruby ($n = 52$, median $135,000).

Ultimately, moving away from commoditized skill sets toward differentiated expertise—whether in specialized architecture, advanced domain execution, or technical management ($n = 26$, median $202,500), consistently yields significant economic rewards.

## 
## --- Compensation Premiums by Role ---

## # A tibble: 10 × 4
##    DevType                                Respondent_Count Median_Comp Mean_Comp
##    <chr>                                             <int>       <dbl>     <dbl>
##  1 Engineering manager                                  26      202500  1245990.
##  2 DevOps engineer or professional                      17      144700   253188.
##  3 Data engineer                                        18      137500   337667.
##  4 Architect, software or solutions                     54      131250   320564.
##  5 Applied scientist                                    11      130000  1522273.
##  6 Other (please specify):                              20      120000  1102374.
##  7 Developer, back-end                                 137      114038   665147.
##  8 Developer, full-stack                               230      100000   526223.
##  9 Developer, embedded applications or d…               22       96500   378362.
## 10 Senior executive (C-suite, VP, etc.)                 16       95000   130824.

## 
## --- Compensation Premiums by Language/Skill ---

## # A tibble: 10 × 4
##    Language            User_Count Median_Comp Mean_Comp
##    <chr>                    <int>       <dbl>     <dbl>
##  1 Go                         117      138000   633368.
##  2 Ruby                        52      135000   511443.
##  3 MATLAB                      26      120000  1258353.
##  4 C++                        159      115000   818807.
##  5 Kotlin                      79      115000   605684.
##  6 Visual Basic (.Net)         31      115000   140823.
##  7 Java                       216      114019   728760.
##  8 Python                     417      113000   587519.
##  9 SQL                        460      112500   438357.
## 10 C                          127      110000   408850.

Step E: Job Satisfaction & Organizational Impact Analysis

This step cross-references JobSat (job satisfaction scores) with compensation and specialization to examine whether professionals tackling complex problem-solving report higher career leverage and satisfaction.

The job satisfaction and compensation metrics further confirm that making professional differentiation is essential for maximizing earning potential. While high-supply generalist roles like full-stack developers ($n = 216$, median $100,000$) and front-end developers ($n = 29$, median $80,000$) occupy lower compensation tiers—with front-end developers also reporting the lowest job satisfaction (6.34)—differentiated roles command significant market premiums. Specialized technical roles such as data engineers ($n = 15$, median $125,000$) and solutions architects ($n = 47$, median $120,000$) leverage domain scarcity and complex system design to secure higher pay and top-tier job satisfaction (7.70). Most dramatically, engineering managers ($n = 20$) leverage leadership and strategic oversight to achieve the dataset’s highest median compensation at $202,500—more than double the baseline for commoditized execution roles.

## 
## --- Job Satisfaction & Compensation by Role ---

## # A tibble: 9 × 4
##   DevType                                       Avg_Job_Sat Median_Comp Count
##   <chr>                                               <dbl>       <dbl> <int>
## 1 Architect, software or solutions                     7.70      120000    47
## 2 Engineering manager                                  7.65      202500    20
## 3 Developer, embedded applications or devices          7.59       96500    22
## 4 Developer, desktop or enterprise applications        7.45       90000    31
## 5 Developer, back-end                                  7.09      120000   130
## 6 Developer, full-stack                                7.07      100000   216
## 7 Developer, mobile                                    6.91       80000    22
## 8 Data engineer                                        6.87      125000    15
## 9 Developer, front-end                                 6.34       80000    29

Step F: Publication-Ready Visualizations

Generate professional visualizations to embed directly into your Quarto report

## 
## === Top Database Technologies by Median Compensation ===

## # A tibble: 10 × 4
##    Database             User_Count Median_Comp Mean_Comp
##    <chr>                     <int>       <dbl>     <dbl>
##  1 BigQuery                     53      170000   375145.
##  2 Snowflake                    29      150000   231879.
##  3 Supabase                     32      144400   437166.
##  4 Cosmos DB                    33      140000   408617.
##  5 Dynamodb                     83      135000   573930.
##  6 Redis                       211      130000   447317.
##  7 Elasticsearch               128      125500   581050.
##  8 Cloud Firestore              25      120000   242486.
##  9 MongoDB                     129      119000   772091.
## 10 Microsoft SQL Server        226      117000   511977.

## 
## === AI Tool Adoption vs. Compensation ===

## # A tibble: 5 × 4
##   AISelect                                    Count Median_Comp Mean_Comp
##   <chr>                                       <int>       <dbl>     <dbl>
## 1 Yes, I use AI tools daily                     378      115000   702824.
## 2 Yes, I use AI tools monthly or infrequently    96      102500   272462.
## 3 Yes, I use AI tools weekly                    129      100000   266443.
## 4 No, and I don't plan to                       119       99000   735311.
## 5 No, but I plan to soon                         24       90500   335919.

## 
## === Compensation Summary by Workplace Tool Count ===

## # A tibble: 5 × 4
##   ToolBucket              Respondent_Count Median_Comp Mean_Comp
##   <fct>                              <int>       <dbl>     <dbl>
## 1 Low (0-3 tools)                      122       90000   804553.
## 2 Moderate (4-6 tools)                 209      110000   478349.
## 3 High (7-10 tools)                    190      115000   510841.
## 4 Very High (11-20 tools)               90      120000   178666.
## 5 Expert/Max (21+ tools)                14       84000    89242.

The provided output tables clearly demonstrate that technical differentiation, tool adoption, and specialized skill mastery directly correlate with higher median compensation. Looking at database technologies, specialized enterprise data-warehousing platforms like BigQuery ($n = 53$, median $170,000$) and Snowflake ($n = 29$, median $150,000$) command the highest median salaries, whereas widespread legacy systems like Microsoft SQL Server ($n = 226$, median $117,000$) offer significantly lower baseline compensation due to high talent availability. This pattern holds true in workflow adoption: professionals who differentiate themselves by leveraging daily AI tools ($n = 378$, median $115,000$) out-earn non-users ($n = 119$, median $99,000$) and infrequent users, proving that integrating cutting-edge tools boosts market value. Furthermore, the tool-count distribution reinforces that targeted competency pays best; compensation steadily climbs from low tool usage ($0–3$ tools, median $90,000$) up to a sweet spot of $11–20$ tools (median $120,000$). However, excessive tool clutter without depth causes earnings to plummet at the extreme tier ($21+$ tools, median $84,000$), illustrating that true market value comes from mastering a differentiated, high-impact stack rather than becoming a superficial generalist.

Critical Analysis

The provided empirical survey data across roles, programming languages, databases, and tool workflows provides clear evidence that career differentiation, rather than superficial execution, dictates market value in data science and software engineering. Evaluating these results through the lens of strategic impact, problem-solving complexity, and measurable ROI illustrates why certain skills command high premiums while others face wage stagnation.

When analyzing the spectrum of developer roles and tool sets through the framework of Strategic Impact versus Commodity Work, a distinct boundary emerges between standardized execution and high-leverage architectural capability. Commodity execution—represented by front-end developers earning a median of $80,000$ with the lowest job satisfaction of 6.34, or generalist full-stack developers earning $100,000—reflects roles centered on routine, replaceable labor.

These roles focus heavily on baseline implementation, where high talent availability limits individual leverage. Conversely, positions that shift the focus toward high-leverage architectural and operational frameworks—such as Solutions Architects earning $120,000–$131,250$ with top-tier job satisfaction (7.70), DevOps Engineers earning $144,700$, and Engineering Managers commanding $202,500$ demonstrate the immense value of strategic oversight. By moving beyond modular task execution to designing end-to-end system architectures, organizing cloud infrastructure, and bridging technical delivery with organizational strategy, professionals decouple their value from manual coding hours and align it with overall operational capacity.

This dynamic is further reinforced by the dimension of Problem-Solving Complexity and Novelty, which dictates that premium rewards accrue to those capable of navigating unstructured, ambiguous technical environments rather than relying on off-the-shelf automation. In the programming language and database domains, baseline tools like standard SQL ($112,500 median$) and legacy relational databases like Microsoft SQL Server ($117,000 median$) serve as foundational industry standards but offer lower financial returns due to their routine, well-documented nature. In contrast, technologies designed for complex, distributed computing and scalable data architecture—such as Go ($138,000 median$), Ruby ($135,000 median$), BigQuery ($170,000 median), and Snowflake ($150,000 median$)—yield significant market premiums. These technologies require specialized expertise in handling high-throughput data pipelines, complex analytical queries, and cloud-native infrastructure—challenges that standardized, off-the-shelf automated scripts cannot autonomously resolve. The tool usage data underscores this principle through a clear non-linear trend: while integrating productivity enhancers like daily AI tools boosts median compensation to $115,000 (compared to $99,000$ for non-users), attempting to cover surface-level tools without deep mastery leads to severe diminishing returns. Compensation rises steadily as tool mastery expands up to an optimal range of 11–20 tools ($120,000 median$), but drops dramatically to $84,000 for professionals claiming 21 or more tools.

This sudden drop illustrates that superficial familiarity across dozens of disparate technologies lacks the analytical depth required to solve novel, unstructured engineering problems, effectively reducing the practitioner back to a commodity tier. Ultimately, the ultimate driver of differentiation is Value Creation and Measurable ROI, where technical skills directly move enterprise metrics, optimize core operational workflows, or unlock proprietary revenue streams. High-earning roles like Data Engineers ($125,000–$137,500$ median) and Applied Scientists ($130,000$ median, with an extraordinary mean compensation exceeding $1.5 million) sit directly at the intersection of data assets and enterprise decision-making. Rather than producing static dashboards or standard reports, these professionals build proprietary algorithmic pipelines, design scalable storage environments, and deploy advanced models that optimize business workflows and drive commercial outcomes.

Similarly, adopting enterprise cloud platforms like BigQuery and Snowflake allows organizations to process massive datasets in real time, delivering actionable market intelligence that competitors relying on traditional database setups cannot easily match. By anchoring specialized technical skills directly to measurable organizational performance and competitive advantage, differentiated professionals secure both superior compensation and greater career longevity.

Conclusion

The empirical survey data demonstrates that market compensation in technical fields is fundamentally governed by the level of skill differentiation. Professionals engaged in commodity execution—such as front-end development, standard database maintenance, or fragmented tool usage—face strong labor competition and capped earning potential. Maximum economic value and job satisfaction are achieved by professionals who transition toward high-leverage domains, including cloud data engineering, system architecture, specialized backend development, and technical leadership. Ultimately, long-term career growth depends on applying specialized, complex problem-solving capabilities to drive measurable enterprise ROI rather than accumulating superficial familiarity across broad, unintegrated tool sets.

Untitled

Muhammad Imran

2026-10-18