GE143 - 10: Future Data Trends

Related considerations

Dr Robert Batzinger
Instructor Emeritus

2023-10-19

1 New data sources

2 Big data and data analytics

2.1 Data processing

  • Collection
  • Cleaning and standardization
  • Modeling
  • Verification
  • Publication
  • Application

2.2 Windchill data

Wspeed
0 40 35 30 25 20 15 10 5 0 -5 -10 -15 -20 -25 -30 -35 -40 -45
5 36 31 25 19 13 7 1 -5 -11 -16 -22 -28 -34 -40 -46 -52 -58 -63
10 34 27 21 15 11 3 -4 -10 -16 -22 -28 -35 -41 -47 -53 -59 -66 -72
15 32 25 19 13 6 0 -7 -13 -19 -26 -32 -39 -45 -51 -58 -64 -71 -77
20 30 24 17 11 4 -2 -9 -15 -22 -29 -35 -42 -48 -55 -61 -68 -74 -81
25 29 23 16 9 3 -4 -11 -17 -24 -31 -38 -44 -51 -58 -64 -71 -78 -84
30 28 22 15 8 1 -5 -12 -19 -26 -33 -39 -46 -53 -60 -67 -73 -80 -87
35 28 21 14 7 0 -7 -14 -21 -27 -34 -41 -48 -55 -62 -69 -76 -83 -89
40 27 20 13 6 -1 -8 -15 -22 -29 -36 -43 -50 -57 -64 -71 -78 -84 -91
45 26 19 12 5 -2 -9 -16 -23 -30 -37 -44 -51 -58 -65 -72 -79 -86 -93
50 26 19 12 4 -3 -10 -17 -24 -31 -38 -45 -52 -60 -67 -74 -81 -88 -95
55 25 18 11 4 -3 -11 -18 -25 -32 -39 -46 -52 -60 -67 -74 -81 -88 -95
60 25 17 10 3 -4 -11 -19 -26 -33 -40 -48 -55 -62 -69 -76 -84 -91 -98

2.3 Heatmap

2.4 Data conversion

mph temp wchill
0 40 40
0 35 35
0 30 30
0 25 25
0 20 20
0 15 15
0 10 10
0 5 5
0 0 0
0 -5 -5
0 -10 -10
0 -15 -15
0 -20 -20
0 -25 -25
0 -30 -30
0 -35 -35
0 -40 -40
0 -45 -45
5 40 36
10 40 34
15 40 32
20 40 30
25 40 29
30 40 28
35 40 28
40 40 27
45 40 26
50 40 26
55 40 25
60 40 25
5 35 31
10 35 27
15 35 25
20 35 24
25 35 23
30 35 22
35 35 21
40 35 20
45 35 19
50 35 19
55 35 18
60 35 17
5 30 25
10 30 21
15 30 19
20 30 17
25 30 16
30 30 15
35 30 14
40 30 13
45 30 12
50 30 12
55 30 11
60 30 10
5 25 19
10 25 15
15 25 13
20 25 11
25 25 9
30 25 8
35 25 7
40 25 6
45 25 5
50 25 4
55 25 4
60 25 3
5 20 13
10 20 11
15 20 6
20 20 4
25 20 3
30 20 1
35 20 0
40 20 -1
45 20 -2
50 20 -3
55 20 -3
60 20 -4
5 15 7
10 15 3
15 15 0
20 15 -2
25 15 -4
30 15 -5
35 15 -7
40 15 -8
45 15 -9
50 15 -10
55 15 -11
60 15 -11
5 10 1
10 10 -4
15 10 -7
20 10 -9
25 10 -11
30 10 -12
35 10 -14
40 10 -15
45 10 -16
50 10 -17
55 10 -18
60 10 -19
5 5 -5
10 5 -10
15 5 -13
20 5 -15
25 5 -17
30 5 -19
35 5 -21
40 5 -22
45 5 -23
50 5 -24
55 5 -25
60 5 -26
5 0 -11
10 0 -16
15 0 -19
20 0 -22
25 0 -24
30 0 -26
35 0 -27
40 0 -29
45 0 -30
50 0 -31
55 0 -32
60 0 -33
5 -5 -16
10 -5 -22
15 -5 -26
20 -5 -29
25 -5 -31
30 -5 -33
35 -5 -34
40 -5 -36
45 -5 -37
50 -5 -38
55 -5 -39
60 -5 -40
5 -10 -22
10 -10 -28
15 -10 -32
20 -10 -35
25 -10 -38
30 -10 -39
35 -10 -41
40 -10 -43
45 -10 -44
50 -10 -45
55 -10 -46
60 -10 -48
5 -15 -28
10 -15 -35
15 -15 -39
20 -15 -42
25 -15 -44
30 -15 -46
35 -15 -48
40 -15 -50
45 -15 -51
50 -15 -52
55 -15 -52
60 -15 -55
5 -20 -34
10 -20 -41
15 -20 -45
20 -20 -48
25 -20 -51
30 -20 -53
35 -20 -55
40 -20 -57
45 -20 -58
50 -20 -60
55 -20 -60
60 -20 -62
5 -25 -40
10 -25 -47
15 -25 -51
20 -25 -55
25 -25 -58
30 -25 -60
35 -25 -62
40 -25 -64
45 -25 -65
50 -25 -67
55 -25 -67
60 -25 -69
5 -30 -46
10 -30 -53
15 -30 -58
20 -30 -61
25 -30 -64
30 -30 -67
35 -30 -69
40 -30 -71
45 -30 -72
50 -30 -74
55 -30 -74
60 -30 -76
5 -35 -52
10 -35 -59
15 -35 -64
20 -35 -68
25 -35 -71
30 -35 -73
35 -35 -76
40 -35 -78
45 -35 -79
50 -35 -81
55 -35 -81
60 -35 -84
5 -40 -58
10 -40 -66
15 -40 -71
20 -40 -74
25 -40 -78
30 -40 -80
35 -40 -83
40 -40 -84
45 -40 -86
50 -40 -88
55 -40 -88
60 -40 -91
5 -45 -63
10 -45 -72
15 -45 -77
20 -45 -81
25 -45 -84
30 -45 -87
35 -45 -89
40 -45 -91
45 -45 -93
50 -45 -95
55 -45 -95
60 -45 -98

2.5 Replotted data

2.6 5 V’s of Big Data

  • Volume
  • Velocity
  • Variability
  • Veracity
  • Value

2.7 Statistical computing and Dashboarding

  • Customer profiling: https://analytics.google.com/analytics/web/?utm_source=marketingplatform.google.com

  • Customer experience/behavior tracking and sentiment analysis: https://www.australia.com

  • contact tracing:

thaichana

2.8 Search Engine Services components

  • Spider app visits websites to capture content
  • Analysis checks page for
    • structure
    • authority
    • key points
    • links
  • User interface to accept queries to database

2.9 Search Engine Optimization

Improving the chances of finding stuff on the net by understanding the search algorithm priority

  • Looking matching key concepts
  • Evaluating the goodness of fit of the concepts
  • Matches to ad words
  • Organizing the results by popularity

2.10 SEO Example

280 million pages, (650 msec)

  1. Searching and sorting large data set
  2. Big Data Search: Sorting randomness
  3. Filtering and sorting search results

293 million pages, (430 msec)

  1. Searching and sorting large data set
  2. Methods for Sorting Big Data
  3. Filtering and sorting search results
  • 329 million pages, (730 msec)
  1. Filtering and sorting search results
  2. What is Data Sorting?
  3. Methods for Sorting Big Data

2.10.4 When I do a big data search, how do I sort the results?

  • 381 million pages, 770 msec
  1. Methods for Sorting Big Data
  2. Filtering and sorting search results
  3. What is Data Sorting?

2.11 Pitfalls in searchs

  1. Biases from word usage

    • popular triggers
    • ambiguity
    • sarcasm
    • overused sloguns and jargon

  2. Misunderstanding

    • Abbreviations
    • professional jargon
    • minority sense of words

2.12 Obsolences of the web

  • Revision of webpages
  • Closure/restructuring of companies
  • Removal of material due to memory constraints

3 Quantum computing

4 QBits

3.1 Nuclear physics behind Quantum computing

  • Planck: energy is quantized
  • Einstein: photoelectric effect and electron entangement
  • Bohr and Rutherford: quantized energy applications to electron orbits,
  • Louis de Broglie: electron particles also have wave properties
  • Schrödinger: probabilities of energy states of an electron.

3.2 Quantum Computing via Qiskit

  • requires intepreting a problem into the quantum states of a QBit
  • adjusting the quantum states according to basic QBit operations
  • reading the resulting state of the QBit
  • Outputing the results
  • Python is used as the programming language

3.3 Differences between digital and quantum computing

Quality Digital Quantum
Basic unit of memory Bit QBit
Range of values of basic unit 0/1 0…1
Calculations Sequential Simulataneous
Arithmetric Individual numbers Matrixes, vectors
Telecommunications speed of light instaneous
Environment Room temp Cryogenic

3.4 Applications of quantum computing

  • cryptanalysis
  • secure communications
  • prediction of materials properties
  • spectroscopy
  • probability

Quantum computer

4 Blockchain

https://youtu.be/b2bdGEqPmCI?si=1JH9HePyI_PZ-kU5 https://youtu.be/itY6VWpdECc?si=DcuH4PbHUsVp2nr8

PYU GE143 2022/1: Science and IT for the Quality of Life [8]