[et_pb_section fb_built=”1″ admin_label=”Header” _builder_version=”4.27.3″ _module_preset=”default” custom_padding=”0px||0px||false|false” global_colors_info=”{}”][et_pb_row column_structure=”1_2,1_2″ _builder_version=”4.27.3″ _module_preset=”default” custom_padding=”0px|||||” global_colors_info=”{}”][et_pb_column type=”1_2″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_image src=”https:\/\/akosgombkoto.info\/wp-content\/uploads\/2025\/01\/data-science-070-2.png” title_text=”data-science-070-2″ _builder_version=”4.27.4″ _module_preset=”default” max_width_tablet=”500px” max_width_phone=”220px” max_width_last_edited=”off|tablet” max_height_tablet=”200px” max_height_phone=”100px” max_height_last_edited=”on|phone” custom_margin=”|||-8vw|false|false” custom_margin_tablet=”|||0vw|false|false” custom_margin_phone=”” custom_margin_last_edited=”on|tablet” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″][\/et_pb_image][\/et_pb_column][et_pb_column type=”1_2″ _builder_version=”4.27.3″ _module_preset=”default” custom_padding=”||||false|false” custom_padding_tablet=”0px||||false|false” custom_padding_last_edited=”off|desktop” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.3″ _module_preset=”d72c0383-6487-4f2c-ac5c-7d48a6376757″ header_font=”Roboto Slab||||||||” header_text_color=”#000000″ header_font_size=”46px” header_line_height=”1.2em” custom_margin=”||10px||false|false” header_font_size_tablet=”40px” header_font_size_phone=”20px” header_font_size_last_edited=”on|phone” locked=”off” global_colors_info=”{}”]<\/p>\n
<\/p>\n
[\/et_pb_text][et_pb_text _builder_version=”4.27.4″ _module_preset=”default” text_font=”Roboto Mono||||||||” text_text_color=”#fcb03a” text_font_size=”18px” text_line_height=”1.8em” background_color=”#042f4f” custom_padding=”15px||15px|20px|true|false” global_module=”471″ saved_tabs=”all” global_colors_info=”{}”]<\/p>\n
Scroll down to the first section for industrial and e.g. HR application examples.<\/strong><\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/akosgombkoto.info\/wp-content\/uploads\/2024\/11\/heatmap.jpg” title_text=”heatmap” show_in_lightbox=”on” _builder_version=”4.27.3″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ admin_label=”Blog” _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_row _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.3″ _module_preset=”default” text_font=”Roboto Mono||||||||” text_text_color=”#fcb03a” text_font_size=”18px” text_line_height=”1.8em” background_color=”#042f4f” custom_padding=”15px||15px|20px|true|false” global_colors_info=”{}”]<\/p>\n Industrial and e.g. HR Application Examples\n(Non-industrial examples, such as HR, are located below the industrial example block.):<\/strong><\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.27.4″ _module_preset=”default” text_font=”Roboto Mono||||||||” text_text_color=”#FFFFFF” text_font_size=”16px” text_line_height=”1.8em” background_color=”#042f4f” custom_padding=”20px|20px|20px|20px|true|true” global_colors_info=”{}”]<\/p>\n Industrial Example:<\/p>\n The Meaning and Usefulness of Correlation (Exploratory Data Analysis \u2013 EDA) in Searching for Industrial Relationships: In focus with correlation analysis The Role of Correlation Analysis in EDA: Unveiling Relationships Between Data:<\/p>\n During EDA, correlation helps reveal how variables influence each other. For example, in a manufacturing database, examining the relationship between machine temperature and product quality can help determine whether high temperatures lead to poor quality.<\/p>\n Detecting Trends and Patterns:<\/p>\n Correlation analysis can uncover trends and patterns. If two variables strongly correlate, they may move together, indicating a shared effect or cause.<\/p>\n Recognizing Multicollinearity:<\/p>\n Multicollinearity occurs when two or more independent variables strongly correlate. This can pose challenges in regression modeling, as it may distort results. Correlation analysis in EDA helps identify such issues. Correlation aids in the initial understanding of data and lays the groundwork for further analysis, such as regression modeling, where the goal might be to predict one variable based on others.<\/p>\n Data Cleaning:<\/p>\n Correlation analysis can also assist in data cleaning. If two variables strongly correlate, removing one from further analysis may reduce redundancy and complexity.<\/p>\n Practical Example: Temperature vs. Product Quality A correlation matrix and visualizations (e.g., heatmaps) can quickly highlight the most significant factors to consider for optimizing production.<\/p>\n Correlation analysis in EDA is a powerful tool for uncovering relationships in data, detecting anomalies, and preparing for deeper analysis. Properly conducted correlation analysis can provide a solid foundation for predictive models and other statistical analyses.<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.27.3″ _module_preset=”default” text_font=”Roboto Mono||||||||” text_text_color=”#FFFFFF” text_font_size=”16px” text_line_height=”1.8em” background_color=”#042f4f” custom_padding=”20px|20px|20px|20px|true|true” global_colors_info=”{}”]<\/p>\n HR Example:<\/p>\n 1. Training Programs and Performance Improvement: 2. Workplace Atmosphere and Sick Leave: [\/et_pb_text][et_pb_toggle title=”Korrel\u00e1ci\u00f3s Heatmap Magyarorsz\u00e1g gazdas\u00e1gi szektorair\u00f3l (leny\u00edl\u00f3 tartalom – kattints a jobb sz\u00e9ls\u0151 + ikonra)” open_toggle_text_color=”#fcb03a” open_toggle_background_color=”#042f4f” closed_toggle_text_color=”#fcb03a” closed_toggle_background_color=”#042f4f” icon_color=”#fcb03a” open_icon_color=”#fcb03a” _builder_version=”4.27.4″ _module_preset=”default” title_font_size=”18px” title_line_height=”1.6em” closed_title_font_size=”18px” body_text_color=”#FFFFFF” body_font_size=”16px” body_line_height=”1.8em” global_colors_info=”{}”]<\/p>\n Based on data from the UNIDO website, the correlation matrix shows the relationships between the output values of Hungary's economic sectors.\n\nA \"heatmap\" provides an excellent basis for identifying basic relationships in exploratory data analysis (EDA).\nHowever, we must understand the concept of correlation and apply critical thinking!\nThe chart has a flaw: time \u2013 years do not correlate with anything (despite the chart showing different values), as time was the variable along which the data was collected. This is why it is indicated in the matrix.<\/p>\n A correlation heatmap is one of the best tools in Exploratory Data Analysis (EDA) for visualizing relationships.<\/p>\n [\/et_pb_toggle][et_pb_toggle title=”Korrel\u00e1ci\u00f3 \u00e9s kritikus gondolkod\u00e1s: (leny\u00edl\u00f3 tartalom – kattints a jobb sz\u00e9ls\u0151 + ikonra)” open_toggle_text_color=”#fcb03a” open_toggle_background_color=”#042f4f” closed_toggle_text_color=”#fcb03a” closed_toggle_background_color=”#042f4f” icon_color=”#fcb03a” open_icon_color=”#fcb03a” _builder_version=”4.27.3″ _module_preset=”default” title_font_size=”18px” title_line_height=”1.6em” closed_title_font_size=”18px” body_text_color=”#FFFFFF” body_font_size=”16px” body_line_height=”1.8em” global_colors_info=”{}”]<\/p>\n Correlation measures the strength and direction of the relationship between two or more variables. Mathematically, it shows how one variable changes with another. The most common form is linear correlation, where the relationship between variables can be represented by a straight line.<\/p>\n The correlation coefficient (usually denoted as r) expresses the strength and direction of the relationship:<\/p>\n r = +1: Perfect positive correlation (both variables increase together).\nr = -1: Perfect negative correlation (one increases as the other decreases).\nr = 0: No linear correlation (no obvious relationship between the variables).\n\nPearson's correlation coefficient is the most commonly used, but there are other types, such as Spearman rank correlation.<\/p>\n Why is Critical Thinking Important When Interpreting Correlation?<\/strong> Correlation Does Not Imply Causation:\nCorrelation does not mean that one variable causes the other. Two variables may correlate closely without a causal relationship. For example, research shows a correlation between frozen food consumption and divorce rates, but this does not imply a causal link.<\/p>\n Third Variable (Confounding Factor):\nSometimes the relationship between two variables arises due to a third variable. For example, there might be a positive correlation between ice cream sales and sunburn, but the actual relationship is likely due to summer weather (the confounding factor), which drives both activities.<\/p>\n Correlation Traps:\nThe value of the correlation coefficient can be misleading if we don\u2019t understand the context. For instance, a non-linear relationship may not show up well with Pearson\u2019s coefficient, even if a strong connection exists.<\/p>\n Sample Size:\nSmall sample sizes can distort the correlation coefficient and fail to reflect the true relationship in the entire population. Ensuring a sufficiently large sample size is crucial for statistical validity.<\/p>\n Effect of Outliers:\nExtreme values (outliers) can significantly impact the correlation coefficient, as Pearson\u2019s correlation is sensitive to them. A few outliers can increase variance and skew the correlation.<\/p>\n Summary:\nCorrelation is a valuable tool for exploring relationships between variables, but it is crucial to understand that correlation does not imply causation. Correlation analyses should always be interpreted critically, considering possible confounding factors, sample size, and data type.<\/p>\n [\/et_pb_toggle][et_pb_toggle title=”K\u00f3dmag: (leny\u00edl\u00f3 tartalom – kattints a jobb sz\u00e9ls\u0151 + ikonra)” open_toggle_text_color=”#fcb03a” open_toggle_background_color=”#042f4f” closed_toggle_text_color=”#fcb03a” closed_toggle_background_color=”#042f4f” icon_color=”#fcb03a” open_icon_color=”#fcb03a” _builder_version=”4.27.3″ _module_preset=”default” title_font_size=”18px” title_line_height=”1.6em” closed_title_font_size=”18px” body_text_color=”#FFFFFF” body_font_size=”16px” body_line_height=”1.8em” global_colors_info=”{}”]<\/p>\n # CSV f\u00e1jl beolvas\u00e1sa (cser\u00e9ld ki a f\u00e1jl el\u00e9r\u00e9si \u00fatj\u00e1t)<\/p>\n file_path = r'C:\\\\Users\\\\Akos\\\\Desktop\\\\Python\\\\UNIDO_MO_Gazdas\u00e1ga\\\\Mo_gazdasaga_korrelacio_csv_01.csv' # A f\u00e1jl el\u00e9r\u00e9si \u00fatja<\/p>\n data = pd.read_csv(file_path)<\/p>\n # Korrel\u00e1ci\u00f3s m\u00e1trix kisz\u00e1m\u00edt\u00e1sa<\/p>\n corr_matrix = data.corr()<\/p>\n # Heatmap k\u00e9sz\u00edt\u00e9se<\/p>\n plt.figure(figsize=(16, 12))<\/p>\n sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', fmt='.2f', linewidths=0.5)<\/p>\n plt.title(\"Part of Correlation Matrix of Hungary's Economic Sectors (Heatmap) from 2005 to 2021: source: UNIDO\")<\/p>\n plt.subplots_adjust(top=0.97)<\/p>\n plt.subplots_adjust(bottom=0.36)<\/p>\n plt.subplots_adjust(left=0.215, right=1.0)<\/p>\n plt.show()[\/et_pb_toggle][et_pb_text _builder_version=\"4.27.3\" _module_preset=\"default\" text_font=\"Roboto Mono||||||||\" text_text_color=\"#fcb03a\" text_font_size=\"18px\" text_line_height=\"1.8em\" background_color=\"#042f4f\" custom_padding=\"15px||15px|20px|true|false\" global_colors_info=\"{}\"]<\/p>\n Data source:<\/strong><\/p>\n [\/et_pb_text][et_pb_image src=\"https:\/\/akosgombkoto.info\/wp-content\/uploads\/2024\/11\/5_Adatforras_UNIDOKepernyokep-2024-11-09-235155.png\" title_text=\"5_Adatforras_UNIDOK\u00e9perny\u0151k\u00e9p 2024-11-09 235155\" _builder_version=\"4.27.3\" _module_preset=\"default\" global_colors_info=\"{}\"][\/et_pb_image][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>","protected":false},"excerpt":{"rendered":" Korrel\u00e1ci\u00f3s Heatmap Magyarorsz\u00e1g gazdas\u00e1gi szektorair\u00f3l Az ipari \u00e9s p\u00e9ld\u00e1ul HR-es alkalmaz\u00e1si p\u00e9ld\u00e1\u00e9rt g\u00f6rgess lejjebb az els\u0151 bekezd\u00e9sre. Ipari \u00e9s p\u00e9ld\u00e1ul HR-es alkalmaz\u00e1si p\u00e9ld\u00e1k (nem ipari p\u00e9ld\u00e1k mint p\u00e9ld\u00e1ul a HR-es az ipari p\u00e9lda blokk alatt tal\u00e1lhat\u00f3):Ipari p\u00e9lda: A korrel\u00e1ci\u00f3 (felt\u00e1r\u00f3 adatelemz\u00e9s – EDA) \u00e9rtelme \u00e9s haszn\u00e1lhat\u00f3s\u00e1ga ipari \u00f6sszef\u00fcgg\u00e9sek keres\u00e9se eset\u00e9n:A fet\u00e1r\u00f3 adatelemz\u00e9s (Exploratory Data Analysis, […]<\/p>","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"class_list":["post-114","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/pages\/114","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/comments?post=114"}],"version-history":[{"count":53,"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/pages\/114\/revisions"}],"predecessor-version":[{"id":682,"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/pages\/114\/revisions\/682"}],"wp:attachment":[{"href":"https:\/\/akosgombkoto.info\/en\/wp-json\/wp\/v2\/media?parent=114"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Exploratory Data Analysis (EDA) aims to visualize, understand, and draw preliminary conclusions from data without predefined hypotheses. During EDA, we explore data patterns, trends, and potential relationships.<\/p>\n
Correlation analysis is a key tool in EDA that determines whether there is a relationship between two or more variables. Correlation indicates the strength and direction of a relationship: it can be positive (both variables increase together), negative (one increases as the other decreases), or absent (the variables are independent).<\/p>\n
Laying the Foundation for Further Analysis:<\/p>\n
Suppose we are working with production data from a manufacturing plant and want to identify factors that most influence product quality. During EDA, we examine the correlation between different variables, such as:<\/p>\n
Pressure vs. Product Quality
Speed vs. Product Quality<\/p>\n
Correlation analysis can determine whether employee performance improves after participating in various training programs. For example, it can examine the correlation between training hours and annual performance evaluation results. A positive correlation suggests that training contributes to improved performance, making it worthwhile to offer additional training.<\/p>\n
The relationship between workplace atmosphere (e.g., relationships with colleagues) and sick leave data can also be analyzed. A strong correlation between negative workplace relationships and frequent sick leave could indicate the need to improve the workplace environment through team-building activities or conflict resolution training.<\/p>\n
While correlation is a valuable tool for studying relationships between variables, it\u2019s essential to interpret it critically because the results can be misleading for several reasons:<\/p>\nimport pandas as pd
\nimport seaborn as sns
\nimport matplotlib.pyplot as plt<\/p>\n