Introduction

This report explores insights from the US Permanent Visa Applications dataset, shedding light on application types and the distribution of applicant nationalities. By employing various data visualization techniques, it aims to uncover key findings. Notably, the dataset presents a challenge due to significant popularity disparities among certain nationalities or visa types, necessitating focused visualizations for comprehensive analysis.

Dataset

The US Permanent Visa Applications dataset, compiled by the US Department of Labor, spans from 2012 to 2017. It comprises comprehensive details including employer information, offered wages, job history, employee education, previous visa records, legal representation, and final decisions. The dataset selection stemmed from personal relevance as a current F-1 visa holder with aspirations for an H-1B visa in the future, making it particularly intriguing.

Findings

Working with this dataset proved engaging, offering a plethora of variables to explore and specific data segments to analyze. Further investigation into legislative changes or geopolitical events during the dataset’s timeframe could provide valuable context, potentially uncovering hidden variables influencing visa application trends.

Upon initial examination, the top 10 countries of citizenship for visa applicants emerged as India, China, South Korea, Canada, the Philippines, Mexico, the United Kingdom, Taiwan, Pakistan, and France. These countries demonstrated an upward trend in US visa requests over time. Notably, the most sought-after visa categories included H, F, L, E, and B, with variations observed within each category regarding specific visa class popularity. Among these, the H-1B visa emerged as the most frequently sought-after type.

Data Exploration

The first section presents an initial exploration of the dataset, providing insight into its size and guiding the direction for subsequent visualizations. I gained some interesting information about the size of the dataset, the countries which have the most visa applicants, and some standout columns for later visualizations.

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import plotly.express as px
import warnings

warnings.filterwarnings("ignore")

path = "/Users/thomasgray/Library/CloudStorage/OneDrive-Personal/Python/PythonProject/"
filename = "us_perm_visas.csv"

df = pd.read_csv(path + filename, usecols=['case_status', 'country_of_citizenship'], low_memory=False)

# Get the number of rows and columns
num_rows, num_cols = df.shape

# Display the number of rows and columns in a sentence
print(f"This data set has {num_rows} rows and {num_cols} columns.")
## This data set has 374362 rows and 2 columns.
# Group by country of citizenship and count the number of applications
country_counts = df['country_of_citizenship'].value_counts().head(10)

# Print the top 10 countries for visa applicants
print("Top 10 Countries for Visa Applicants:")
## Top 10 Countries for Visa Applicants:
for country in country_counts.index.tolist():
    print(country)
## INDIA
## CHINA
## SOUTH KOREA
## CANADA
## MEXICO
## PHILIPPINES
## UNITED KINGDOM
## TAIWAN
## PAKISTAN
## FRANCE
df = pd.read_csv(path + filename, usecols=['case_status', 'country_of_citizenship', 'case_received_date',
                                           'class_of_admission', 'employer_state', 'employer_city'],
                 low_memory=False)

# Count non-zero columns for specified column names
non_zero_counts = df[['case_status', 'country_of_citizenship', 'case_received_date',
                      'class_of_admission', 'employer_state', 'employer_city']].notna().sum()

print("Columns of Interest:")
## Columns of Interest:
print(non_zero_counts)
## case_status               374362
## country_of_citizenship    353729
## case_received_date        239091
## class_of_admission        351517
## employer_state            374320
## employer_city             374348
## dtype: int64

Visualization 1

Case Status by Country

The initial visualization showcases the breakdown of case status types, indicating whether visa applications were certified or denied. With nearly 200 countries represented in the dataset, the focus narrows to the top 10 countries with the highest application volumes.

Given India’s substantial application volume relative to the rest of the dataset, its data is presented separately in a distinct graph below, enhancing clarity and ease of interpretation.

The plots unveil that only a small fraction of applications were denied within this subset of countries. However, the ratio of certified to denied applications varies among different countries. While this difference could be attributed to variations in visa types or discrepancies in the number of applicants per country, drawing precise conclusions based solely on this plot remains challenging.

df = pd.read_csv(path + filename, usecols=['case_status', 'country_of_citizenship'], low_memory=False)

# Group by country of citizenship and count the number of visa applications
country_counts = df.groupby('country_of_citizenship').size().sort_values(ascending=False).head(10).index

country_counts_without_india = country_counts[country_counts != 'INDIA']
df_top_9 = df[df['country_of_citizenship'].isin(country_counts_without_india)]
df_india = df[df['country_of_citizenship'] == 'INDIA']

# Group by country of citizenship and case status, count the number of visa applications
country_counts_with_status = df_top_9.groupby(['country_of_citizenship', 'case_status']).size().unstack(fill_value=0)

# Plot a grouped bar chart for top 9 countries
plt.figure(figsize=(12, 8))
country_counts_with_status.plot(kind='bar', stacked=True, colormap='viridis', ax=plt.gca())
plt.title('Visa Applications by Country and Case Status (Top 9 Countries)')
plt.xlabel('Country of Citizenship')
plt.ylabel('Number of Visa Applications')
plt.xticks(rotation=45, ha='right')
plt.legend(title='Case Status')
plt.tight_layout()
plt.show()

# Group by case status
country_counts_india = df_india.groupby('case_status').size()

# Plot a bar chart for India's case status breakdown
plt.figure(figsize=(8, 6))
sns.barplot(x=country_counts_india.index, y=country_counts_india.values, palette='viridis')
plt.title('Visa Applications from India by Case Status')
plt.xlabel('Case Status')
plt.ylabel('Number of Visa Applications')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()

Visualization 2

Number of Visa Applications by Year and Country of Citizenship

The visualization presented here depicts the variation in visa application figures over time, segmented by the citizenship country, with a spotlight on the top nine countries following India. A prevalent trend among the majority of listed countries is the sustained growth in application numbers in recent years. Particularly noteworthy is the substantial surge in applications from China and South Korea, outpacing that of other countries by a significant margin. Delving into the specific visa types sought after could unveil valuable insights into the factors driving this notable increase in these particular countries.

df = pd.read_csv(path + filename, low_memory=False)

# Exclude data for India
df_filtered = df[df['country_of_citizenship'] != 'India']

# Get the top 10 countries
top_countries = df_filtered['country_of_citizenship'].value_counts().nlargest(11).index[1:]  # Excluding India

# Filter the DataFrame to include only the top 10 countries
df_filtered = df_filtered[df_filtered['country_of_citizenship'].isin(top_countries)]

# Convert case_received_date to datetime format and extract the year
df_filtered['year'] = pd.to_datetime(df_filtered['case_received_date']).dt.year

# Filter the DataFrame to include years from 2012 to 2016
df_filtered = df_filtered[df_filtered['year'].between(2012, 2016)]

# Group by year and country of citizenship, count the number of visa applications
country_counts_by_year = df_filtered.groupby(['year', 'country_of_citizenship']).size().reset_index(name='count')

# Plot a line graph
plt.figure(figsize=(12, 8))
sns.lineplot(data=country_counts_by_year, x='year', y='count', hue='country_of_citizenship', palette='bright', marker='o', linewidth=2)
plt.title('Visa Applications by Country and Year (2012-2016 Top 10 Countries Excluding India)')
plt.xlabel('Year')
plt.ylabel('Number of Visa Applications')
plt.xticks(range(2012, 2017))  # Set x-axis ticks to every year
plt.xticks(rotation=45)
plt.legend(title='Country of Citizenship', bbox_to_anchor=(1.05, 1), loc='upper left')
plt.tight_layout()
plt.show()

Visualization 3

The Different Visa Types Applied For

The previous visualizations underscored the importance of delving deeper into specific visa categories to gain a more comprehensive understanding of the dataset. Thus, this visualization endeavors to shed light on the distribution of the most prevalent visa types.

The primary categories encompass H (temporary worker), F (students), L (foreign employees of US corporations), E (treaty investor), and B (temporary entry). Notably, the ‘H’ category emerges as the most predominant, further subdivided into nine distinct visa classes, with the H-1B visa emerging as the prevailing choice overall.

df = pd.read_csv(path + filename, usecols=['class_of_admission'], low_memory=False)

# Define the visa types to include in the pie chart
visa_types = ['H', 'F', 'L', 'E', 'B']

# Filter out rows where 'class_of_admission' is not in the specified visa types
df_filtered = df[df['class_of_admission'].str[0].isin(visa_types)]

# Group by visa category and count the number of applications
visa_category_counts = df_filtered['class_of_admission'].str[0].value_counts()

# Pie Chart Creation
fig, ax = plt.subplots(figsize=(8, 8))

colors = plt.cm.tab10.colors

wedges_outer, texts_outer, autotexts_outer = ax.pie(visa_category_counts, labels=None, autopct='%1.1f%%', startangle=140, colors=colors, wedgeprops=dict(width=0.7, edgecolor='w'))

ax.pie([1], radius=0.2, colors=['w'], wedgeprops=dict(width=0.2, edgecolor='w'))
for autotext in autotexts_outer:
    autotext.set_color('white')
    autotext.set_fontsize(10)
    autotext.set_bbox(dict(facecolor='black', alpha=0.7, edgecolor='none', pad=0.5))

total_count = visa_category_counts.sum()
plt.text(0, 0, f"Total: {total_count}", ha='center', fontsize=12)
plt.title('Most Popular Visa Types Pie Chart')

legend_labels = [f"{label} ({value})" for label, value in visa_category_counts.items()]
plt.legend(legend_labels, loc='upper center', bbox_to_anchor=(0.5, 0.22), shadow=True, ncol=2)
plt.axis('equal')
plt.show()

Visualization 4

Map of Employers Hiring Foreign Workers

Continuing from the previous visualization, I delved into examining the employers hiring applicants with H visas and their geographical distribution. The resulting map provides a fascinating insight into the locations where visa applicants find employment. As anticipated, states boasting major metropolitan hubs such as California, Washington, New York, and Texas recorded the highest numbers of visa-sponsored employment opportunities.

df = pd.read_csv("us_perm_visas.csv")

# Create a dictionary to map state names to abbreviations
state_name_to_abbrev = {
    'ALABAMA': 'AL', 'ALASKA': 'AK', 'ARIZONA': 'AZ', 'ARKANSAS': 'AR', 'CALIFORNIA': 'CA',
    'COLORADO': 'CO', 'CONNECTICUT': 'CT', 'DELAWARE': 'DE', 'FLORIDA': 'FL', 'GEORGIA': 'GA',
    'HAWAII': 'HI', 'IDAHO': 'ID', 'ILLINOIS': 'IL', 'INDIANA': 'IN', 'IOWA': 'IA',
    'KANSAS': 'KS', 'KENTUCKY': 'KY', 'LOUISIANA': 'LA', 'MAINE': 'ME', 'MARYLAND': 'MD',
    'MASSACHUSETTS': 'MA', 'MICHIGAN': 'MI', 'MINNESOTA': 'MN', 'MISSISSIPPI': 'MS', 'MISSOURI': 'MO',
    'MONTANA': 'MT', 'NEBRASKA': 'NE', 'NEVADA': 'NV', 'NEW HAMPSHIRE': 'NH', 'NEW JERSEY': 'NJ',
    'NEW MEXICO': 'NM', 'NEW YORK': 'NY', 'NORTH CAROLINA': 'NC', 'NORTH DAKOTA': 'ND', 'OHIO': 'OH',
    'OKLAHOMA': 'OK', 'OREGON': 'OR', 'PENNSYLVANIA': 'PA', 'RHODE ISLAND': 'RI', 'SOUTH CAROLINA': 'SC',
    'SOUTH DAKOTA': 'SD', 'TENNESSEE': 'TN', 'TEXAS': 'TX', 'UTAH': 'UT', 'VERMONT': 'VT',
    'VIRGINIA': 'VA', 'WASHINGTON': 'WA', 'WEST VIRGINIA': 'WV', 'WISCONSIN': 'WI', 'WYOMING': 'WY'
}

# Map full state names to abbreviations in the "employer_state" column
df["employer_state"] = df["employer_state"].map(lambda x: state_name_to_abbrev.get(x, x))

# Count the visa applications for each state
state_counts = df["employer_state"].value_counts()

# Create a new DataFrame with state abbreviations and counts
state_counts_df = pd.DataFrame(state_counts).reset_index()
state_counts_df.columns = ["state_abbrev", "count"]

# Create the choropleth map
fig = px.choropleth(state_counts_df,
                    locations="state_abbrev",
                    locationmode="USA-states",
                    color="count",
                    color_continuous_scale="Plasma",
                    scope="usa")
fig.update_layout(title_text="Number of Visa Applications per State",
                  geo_scope="usa")

Visualization 5

The Top Cities for Visa Applicants

Upon delving deeper into the employment landscape of visa holders, I shifted my focus to a more granular level by examining the data pertaining to the top cities. Employing a bump chart, I unveiled the leading 10 cities for visa holders in employment.

The findings were a blend of expected outcomes and intriguing revelations. While New York’s prominence at the summit didn’t come as a shock, the swift ascent of College Station caught my attention. Given its association with Texas A&M University, this surge in 2015 piqued my curiosity, prompting the need for further investigation into the underlying factors driving this phenomenon.

df = pd.read_csv("us_perm_visas.csv")

df['decision_date'] = pd.to_datetime(df['decision_date'])

# Group by employer city and decision year, count occurrences
city_counts = df.groupby(['employer_city', df['decision_date'].dt.year]).size().reset_index(name='count')

# Pivot the table to have cities as rows and decision years as columns
pivot_table = city_counts.pivot_table(index='employer_city', columns='decision_date', values='count', aggfunc='sum')

# Get top 10 cities with the highest counts
top_10_cities = pivot_table.sum(axis=1).nlargest(10).index

# Filter pivot table for top 10 cities
top_10_pivot = pivot_table.loc[top_10_cities]

# Calculate yearly ranking for each city
rankings = top_10_pivot.rank(ascending=False, method='min', axis=0)

# Plotting the bump chart
plt.figure(figsize=(12, 8))
sns.lineplot(data=rankings.T, dashes=False)
plt.gca().invert_yaxis()
for i, city in enumerate(top_10_pivot.index):
    for year in rankings.columns:
        plt.scatter(year, rankings.loc[city, year], color='black')
plt.xlabel('Year', fontsize=14)
plt.ylabel('Yearly Ranking', fontsize=14)
plt.title('Top 10 Cities by Visa Applications \n Bump Chart', fontsize=16)
plt.yticks(range(1, len(top_10_pivot) + 1))
plt.legend(title='City', bbox_to_anchor=(1.05, 1), loc='upper left')
plt.tight_layout()
plt.show()

Conclusion

In summary, delving into the US Visa applications through this dataset proved to be both enlightening and engaging. Among the 200 countries analyzed, individuals from India emerged as the foremost applicants, closely followed by China and South Korea. The H-1B visa, tailored for specialized temporary workers, stood out as the most sought-after visa type.

Remarkably, the data showcased a consistent upward trend in applications from the top 10 countries, with China and South Korea notably experiencing accelerated growth. California took the lead among all states in employing visa holders, possibly attributed to its bustling tech industry, numerous metropolitan areas, and appealing climate and living conditions. However, further investigation is essential to solidify these observations.

Noteworthy findings also emerged regarding the most popular city for visa holders during the 2011-2016 period. While New York typically held the top spot, an unexpected shift occurred in 2015 when College Station took the lead.

In conclusion, this dataset, along with the visualizations derived from it, provided a comprehensive exploration of Visa Applications, unveiling some unexpected insights along the way.