This article shows some simple queries that help in DWH exploration and a tool that visualizes key columns. The first query shows list of all tables:

/* SAS */
proc sql;
  select *
  from dictionary.tables;
quit;

/* Oracle */
select *
from all_tables

/* MS SQL */
select *
from sys.Tables

The second query shows list of all columns:

/* SAS */
proc sql;
  select *
  from dictionary.columns;
quit;

/* Oracle */
select *
from all_tab_columns

/* MS SQL */
select *
from information_schema.columns

The third example is useful for queries debugging. To make them faster one can limit the number of rows used in a query:

/* SAS */
proc sql;
  select *
  from example(obs = 10);
quit;

/* SAS */
proc sql outobs = 10;
  select *
  from example;
quit;

/* Oracle */
select *
from example
where rownum < 10

/* MS SQL */
select top 10 *
from example

Having a list of columns from the second query one can find columns having same names. Usually key columns in a “good” DWH are named equally. I used R to makes a graph of table links based on column names. Files with data and code can be found in “Download” section.

You will need to install Rgraphviz library as well as windows libraries from graphviz-2.20.3.1.msi. Also make sure that Rgraphviz bin catalog is in your system path variable.

Recently I’ve practiced to make plots in R using data on weight loss competition among members of SQL.ru forum and made a good tutorial. Files with data and code are available in download section. There are detailed comments and commands are intuitive.

One’ll need a forecast library that can be installed using install.packaes(‘forecast’) command or download it from R-project.org and save it in the C:\Program Files\R\R-2.12.2\library\ catalog. Help is available using “?” symbol, for example “?plot”. I’ll give you a bottle of wine if you make the same plot using any other software.

For example in Telecom graphs can be built using phone calls data. The people you call are your first circle. They are relatives, colleagues or friends. You value those people and listen to their opinions. If one of your friends uses mobile internet the telecom operator can offer this service to you with a high probability of purchase.

Social networks like Facebook can find out your first circle using your “friends list” or monitoring the personal pages you visit. The advertising you saw on Facebook could be shown to you because one of your friends clicked on it earlier.

Social Networks are also important in debt collection. The colleagues and friends can influence the debtor and make him pay the debt. This is why banks and collection agencies do actively collect contact information of your friends, neighbors and colleagues. It sometimes happens that the debt is payed by the friends or relatives, not the debtor.

For software companies their experts are the most valued asset. Every person in the company should have access to the expert’s knowledge. The social network graph can show whether the expert is actively helping other colleagues or is he isolated from others. This graph can use data on internal mail and phone conversations.

For example the graph above is based on internet forum SQL.ru → OLAP and DWH. The nodes are the forum members and the edges show whether the member took part in other member’s thread. Every edge has a weight that equals the number of member A posts in member’s B thread plus number of member B posts in member’s A thread.

At first I made a list of all the 3 000+ forum members and added edges. The graph looked like a black spot on a monitor. I removed all the edges with the weights less than 10 and deleted all the members left without edges. That is the last graph in the video. Then I continued to delete edges with a minimal weights till there was only one edge left. That is the first graph in the video. Then I put the graphs in the video in the reverse order, starting with the smallest graph to the biggest one.

The video bellow is based on the forum SQL.ru → Просто треп (just chat).

Consumers with good credit histories can get a new loan with a lower interest rate. But one has to know which CB stores its credit history and what banks can request that history from CB. If your credit history is poor for example you had delinquent loans you better look for a bank that don’t request your credit history.

Russian Central Bank (RCB) web site allows finding the CB’s where ones histories are stored. According to RCB web site there are 800+ credit organizations and 30+ credit bureaus in Russia. Most of the credit histories are stored in the five biggest CB’s: Equifax, Expirian-Interfax, NBKI, Infocredit and MBKI.

Banks don’t like to publish the list of CB’s they work with. But some information is available online. My task was to develop the schema of banks and credit bureaus cooperation. For instance if one sentence contains both bank and credit bureau names it is very probable they do exchange information with each other. But I had to exclude all the sentences containing more than one bank or CB names because that sentence can be a list of some forum participants.

The schema above was developed using 5 000+ html pages that contain at least one of the five biggest CB’s. Unfortunately schema doesn’t show the direction of the information exchange. The bank can request information from one CB and report credit histories to the other. The calculation of data exchange direction is a next task as well as the increase of the number of banks and CB’s.