Welcome to the SAP BPC and HANA blog

Welcome to the SAP BPC and HANA blog On this site, I will publish the contents of my book on Implementing SAP BPC and HANA. It is a compr...

Showing posts with label data science. Show all posts
Showing posts with label data science. Show all posts

sql for Statistics

As part of my sql tutorial which is still very much in progress, I thought I'd do a quick post on using sql with statistics. I am covering two topics today, which are elementary concepts in statistics but forms a compliment my tutorials very well.

Permutations

Consider a set of 4 objects. Now, suppose that we want to fill three positions with objects selected from the 4. We need to consider the order of the items in each set as well, then the number of possible ordered arrangements is 24 and they are


a b c
b a c
c a b
Dab
a b d
 b a d
 c a d
d a c
a c b
 b c a
 c b a
d b c
a c d
 b c d
 c b d
 d b a
a d c
b d a
 c d b
d c a
a d b
b d c
c d a
d c b

The number of possible ordered arrangements can be computed as follows:

Since there are 3 positions and 4 objects, the first position can be filled in 4 different ways. Once the first position is filled the remaining 2 positions can be filled from the remaining 3 objects. Thus, the second position can be filled in 3 ways. The third position can be filled in 2 ways. Then the total number of ways 3 positions can be filled out of 4 objects is given by

(4)(3)(2) = 24.

In HANA, we can write the following sql to achieve the same result set:

do begin

t_data = 
    select 'a' as id
    from dummy
    union all 
    select  'b' as id
    from dummy
    union all 
    select  'c' as id
    from dummy
    union all 
    select  'd' as id
    from dummy
    ;
    
select o.id, t.id, r.id, f.id
from :t_data as o
cross join :t_data as t
cross join :t_data as r
cross join :t_data as f
where o.id != t.id
and o.id != r.id
and o.id != f.id
and t.id != r.id
and t.id != f.id
and r.id != f.id
;

end;


  ;ID;ID;ID;ID
1 ;a ;b ;c ;d 
2 ;a ;b ;d ;c 
3 ;a ;c ;b ;d 
4 ;a ;c ;d ;b 
5 ;a ;d ;b ;c 
6 ;a ;d ;c ;b 
7 ;b ;a ;c ;d 
8 ;b ;a ;d ;c 
9 ;b ;c ;a ;d 
10;b ;c ;d ;a 
11;b ;d ;a ;c 
12;b ;d ;c ;a 
13;c ;a ;b ;d 
14;c ;a ;d ;b 
15;c ;b ;a ;d 
16;c ;b ;d ;a 
17;c ;d ;a ;b 
18;c ;d ;b ;a 
19;d ;a ;b ;c 
20;d ;a ;c ;b 
21;d ;b ;a ;c 
22;d ;b ;c ;a 
23;d ;c ;a ;b 
24;d ;c ;b ;a 



Combinations


In permutation, order is important. But in many problems, the order of selection is not important and interest centers only on the set of r objects. Let c denote the number of subsets of size r that can be selected from n different objects. 

How many committees of two chemists and one physicist can be formed from 4 chemists and 3 physicists?


do begin

t_phys =
    select 'P1' as id
    from dummy
    union all
    select  'P2' as id
    from dummy
    union all
    select  'P3' as id
    from dummy
    ;

t_chem =
    select 'C1' as id
    from dummy
    union all
    select  'C2' as id
    from dummy
    union all
    select  'C3' as id
    from dummy
    union all
    select  'C4' as id
    from dummy
    ;

  
select p.id, ca.id, cb.id
from :t_phys as p
cross join :t_chem as ca
cross join :t_chem as cb
where ca.id != cb.id
  and cb.id > ca.id
order by p.id, ca.id, cb.id
  ;


 ;ID;ID;ID
1 ;P1;C1;C2
2 ;P1;C1;C3
3 ;P1;C1;C4
4 ;P1;C2;C3
5 ;P1;C2;C4
6 ;P1;C3;C4
7 ;P2;C1;C2
8 ;P2;C1;C3
9 ;P2;C1;C4
10;P2;C2;C3
11;P2;C2;C4
12;P2;C3;C4
13;P3;C1;C2
14;P3;C1;C3
15;P3;C1;C4
16;P3;C2;C3
17;P3;C2;C4
18;P3;C3;C4



What is a data scientist?


Data scientists are multi-skilled individuals. They are well educated and technically proficient with in-depth industry knowledge and a sense of curiosity that drives them to solve complex problems.
Data scientists are business analysts, mathematicians, statisticians and computer scientists. These individuals are in high demand and well paid. Who wouldn't want to be one?
Data science as a discipline was virtually unknown a decade ago, but the technological landscape changed and with it the way that companies think about the mass of structured and unstructured data it has accumulated. This data or Big Data is no longer something that can be ignored or something that IT must handle as an afterthought, but to actively mine for insights and increased revenue and the miner of this data is the Data Scientist.
The origin of data scientists
Data scientists started their careers as data analysts, business analysts and statisticians. These individuals are were typically mid to high-level business people with a flair for numbers and the technical skills to access and query data. 
They may also have a strong academic background in Data Science, with many universities offering degrees in Data Science, for example, this one at the University of London.
Duties of a data scientists
There's not a definitive job description when it comes to a data scientist role, but a Data Scientist will have to fulfil a highly technical role while translating and communicating the results of their analysis to exec in their business. Some of the functions may include:
  • Lead data mining and collection procedures
  • Interpret and analyse data problems
  • Conceive, plan and prioritise data projects
  • Build analytic systems and predictive models
  • Test performance of data-driven products
  • Visualise data and create reports


The data scientist’s toolbox
These terms and technologies are commonly used by data scientists:
·         Data visualization: the presentation of data in a pictorial or graphical format so it can be easily analysed.
·         Machine learning: a branch of artificial intelligence based on mathematical algorithms and automation.
·         Deep learning: an area of machine learning research that uses data to model complex abstractions.
·         Pattern recognition: technology that recognizes patterns in data (often used interchangeably with machine learning).
·         Data preparation: the process of converting raw data into another format so it can be more easily consumed.
·         Text analytics: the process of examining unstructured data to glean key business insights.

How do you become a data scientist?
As with the job description of a data scientist, there is no single way to start a career as a data scientist. There are several options though, depending on your interests, the stage of your career, time available and so on. As this blog focus on SAP and its related products, you could try an online course on SAP predictive analytics. This course has been prepared by us to start you off on your Data Scientist journey.
You can access the course here

Here is a preview of the first lecture: