My python final [simple database analysis]

December 19, 2009

My python final project was to write a simple database analysis based on any given data. I thought about it and I decided to have the entire class use the same sets of data. I created a Google form and let them fill them out. I sent out the result in csv and a few other formats, which really made the whole programming easier.

Python has a very nice module called csv. It is already included in 2.6.4 so there is no need to find it. All we need to do is to import it. After a few days fighting with this program, I finally had finished my project.

Here each student is an object. This program is not very complicated. It is very portable. We can control each column at any instance. It is not the best, but good enough to earn an A.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
import csv
class Student(object):
    sports = []
    ftopics = []
    stopics = []
    genders = []
    movieyrs = []
    countrys = []
    def __init__(self,row):
        # we set up the row from the csv data
       self.lname, self.fname, self.ID, self.gender, self.sport, self.movie, self.movieyr, self.country, self.ftopic, self.stopic = row
        # we import each column into our pre-defined lists
       self.sports.append(self.sport)
       self.ftopics.append(self.ftopic)
       self.stopics.append(self.stopic)
       self.genders.append(self.gender)
       self.movieyrs.append(self.movieyr)
       self.countrys.append(self.country)
        # functions to print results anytime we want
    def print_information(self):
       return (self.lname, self.fname, self.ID, self.gender)
    def print_first(self):
       return (self.lname, self.fname, self.sport)
    def print_second(self):
        return (self.lname, self.fname, self.movie, self.movieyr)
    def print_third(self):
        return (self.lname, self.fname, self.country)
    def print_fourth(self):
        return (self.lname, self.fname, self.ftopic, self.stopic)

#########################   THIS IS OUR WELCOME MENU    #########################
#
#               Welcome to CSC 1000 (Fall 2009) - Final Project
#
#
#                           Author: John Yeukhon Wong
#                           E-mail: gokoproject@gmail.com
#
#       Select one of the following to retrieve speific database analysis
#
#           [0]: Basic Information (Last name, first name, ID, gender)
#           [1]: My Favorite Sport (Last name, first name, favorite sport)
#           [2]: My Favorite Movie (Last name, first name, movie title, movie year)
#           [3]: My Favorite Country to Visit (Last name, first name, country name)
#           [4]: My Favorite class topic (Last name, first name, first preference, second preference)
#
#
#           Each item has its own speific analysis result
#           A while = True loop to re-request a new analysis
#           Enter QUIT to leave this program
#
#########################   THIS IS OUR WELCOME MENU    #########################  


# let us initalize some stuff we will use throughout
# we use csv.reader module here to read csv data
choice_list = []
choice_dict = {}

# assuming our csv data in the same path as the python directory
reader = csv.reader(open('new_mondy_csc_data_revise.csv'), delimiter=',', quotechar='"')
header = tuple(reader.next()) # make sure our reading in tuple form

students = list(map(Student, reader)) # read all remaining lines
num_students = len(students) # get the total numbers of students in this survey

def search_function(x):
    if x == 0:
        print "%-17s|%-10s|%-6s|%s" %header[:4] # running header from column 0 to 3
        print "-" * 45
        for student in students:
            print "%-17s|%-10s|%-6s|%3s" % student.print_information()
        print '\n'
    # gender
        for s in set(Student.genders): # class attribute
            print s,":", Student.genders.count(s),"student(s)", "or",round(((float(Student.genders.count(s)) / num_students) *100),1),"%"

    if x == 1: # Basic information
        print '\n' * 2
        print "Students' Sport Preference"
        print '\n'
        print "%-17s|%-10s|%s" %(header[0],header[1],header[4])
        print "-" * 45
        for student in students:
            print "%-17s|%-10s|%s" %student.print_first()
        print '\n'
        # Printing all sports that are specified by students
        for s in set(Student.sports): # class attribute
            print s,":", Student.sports.count(s),"student(s)", "or", round(((float(Student.sports.count(s)) / num_students) *100),1),"%"

        # Printing sports that are not picked
        allsports = ['Basketball','Football','Other','Baseball','Handball','Soccer','Volleyball','I do not like sport']
        for s in set(allsports) - set(Student.sports):
            print s,":", "---", '0%'
        print '\n'
        # Here we list sports most favorite and least favorite
        choice_list = Student.sports
        for choice in choice_list:
            choice_dict[choice] = choice_dict.get(choice, 0) + 1
        print "The MOST favoritable sport is: ", max(choice_dict)
        print "The LEAST favoritable sport is: ", min(choice_dict)
       
    elif x == 2: # My favorite movie
        print "%-17s|%-10s|%-16s|%s" %(header[0],header[1],header[5],header[6])
        print "-" * 45
       
        for student in students:
            print "%-17s|%-10s|%-16s|%s" % student.print_second()
        print '\n'
        # number of old / new movie
        newyear = ['2001','2002','2003','2004','2005','2006','2007','2008','2009']
        counter = 0
        freq = {}
        for i in Student.movieyrs:
            if i not in newyear:
                freq[i] = freq.get(i, 0) + 1
                counter = counter + 1
        print "Numbers of students' favorite movies are OLDER than 2001: ", counter
        print "Numbers of students' favorite movies are NEWER than 2001: ", (num_students - counter)
        print '\n'
    elif x == 3: # My favorite country to visit
        print "%-17s|%-10s|%s" %(header[0],header[1],header[7])
        print "-" * 45
        for student in students:
            print "%-17s|%-10s|%s" %student.print_third()
        print '\n'
        for s in set(Student.countrys):
           print s,":", Student.countrys.count(s),"student(s)", "or", round(((float(Student.countrys.count(s)) / num_students) *100),1),"%"

        choice_list = Student.countrys
        choice_dict = {}
        for choice in set(Student.countrys):
            choice_dict[choice] = choice_dict.get(choice, 0) + 1
        print "The TOP country wanted to visit is: ", max(choice_dict)
       
    elif x == 4: # First and Second class topic preference
        print "%-17s|%-10s|%-15s|%s" %(header[0],header[1],header[8],header[9])
        print "-" * 45
        for student in students:
            print "%-17s|%-10s|%-16s|%s" % student.print_fourth()
        print '\n'
       
        # first topic preference
        print "My first most preferred class topic"
        print "-"*20
        for s in set(Student.ftopics): # class attribute
            print s,":", Student.ftopics.count(s),"student(s)", "or", round(((float(Student.ftopics.count(s)) / num_students) *100),1),"%"

        alltopics = ['Matrices','Probability','Object Oriented Programming','Matlab','Python']
        for s in set(alltopics) - set(Student.ftopics):
            print s, 0, '0%'
           
        choice_list = Student.ftopics
        choice_dict = {}
        for choice in choice_list:
            choice_dict[choice] = choice_dict.get(choice, 0) + 1
        print "The TOP class topic is: ", max(choice_dict)
        print "The LEAST wanted class topic as first preference: ", min(choice_dict)
        print '\n'
       
        print "My second most preferred class topic"
        print "-"*20
        # second topic preference
        for s in set(Student.stopics): # class attribute
            print s,":", Student.stopics.count(s),"student(s)", "or", round(((float(Student.stopics.count(s)) / num_students) *100),1),"%"
        for s in set(alltopics) - set(Student.stopics):
            print s, "---", '0%'
           
        choice_list = Student.stopics
        choice_dict = {}
        for choice in choice_list:
            choice_dict[choice] = choice_dict.get(choice, 0) + 1
        print "Second most TOP class topic is: ", max(choice_dict)
        print "The LEAST wanted class topic of all: ", min(choice_dict)

# Let us bring our menu interface, or instruction to the front

print """
#########################   THIS IS OUR WELCOME MENU    #########################
#
#               Welcome to CSC 1000 (Fall 2009) - Final Project
#
#
#                           Author: John Yeukhon Wong
#                           E-mail: gokoproject@gmail.com
#
#       Select one of the following to retrieve speific database analysis
#
#           [0]: Basic Information (Last name, first name, ID, gender)
#           [1]: My Favorite Sport (Last name, first name, favorite sport)
#           [2]: My Favorite Movie (Last name, first name, movie title, movie year)
#           [3]: My Favorite Country to Visit (Last name, first name, country name)
#           [4]: My Favorite class topic (Last name, first name, first preference, second preference)
#
#
#           Each item has its own speific analysis result
#           A while = True loop to re-request a new analysis
#           Enter QUIT to leave this program
#
#########################   THIS IS OUR WELCOME MENU    #########################
"""
while True:
    x = raw_input("Enter a number to print specific table, or enter QUIT to leave: ")
    print '\n'
    if x == 'QUIT':
        print "Thank you for using our service!"
        break
    search_function(int(x))
    print '\n'

(more...)

Filed under:CSC 1000, Python  Author:John Wong
2 Comments »

First Python program from csc 1000

November 1, 2009

The speaker today gave us a classwork and I did it using modules.

Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import datetime
import time
m = datetime.datetime.now()
k = datetime.date.today()
b = k.isoweekday()
b = time.strftime("%A")
mm = m.month
yy = m.year
dd = m.day
ds = datetime.date(yy,mm,dd)
hh = m.hour
mi = m.minute
ts = datetime.time(hh,mi)
print "It is now",ts.strftime("%H:%M:%p"),b,ds.strftime("%m/%d/%Y")

(more...)

Filed under:CSC 1000, Python  Author:John Wong
No Comments »

First python lecture in csc1000

I didn't have time to write any blog due to heavy college work load. I apologized.
Last week we had our first guest lecture on Python. I enjoyed it because I liked my csc 1000 class. It was fun to do programming.

Here is the lecture presentation:
http://bitrocket.com/csc100/new_notes_1.html

Anyway, python is very different from many conventional languages. It emphasizes on reducing the size of the core and fewer syntax. But keep in mind that usually applications in Python aren't as powerful as those written in C and C++, for example. It is very expensive to write a web application in C because it is very powerful and time consuming. Python is therefore a great candidate. Google uses Python because, it's cheaper. The cost of adding servers is less than the cost of writing the application in C and C++ mainly.

This statement is not entirely true and is very subjective in some aspects. But the overall context, we should agree that Python is consider a powerful programming language.
(more...)

Filed under:CSC 1000, Python  Author:John Wong
No Comments »

working on matrices 3x3 in Matlab (2)

September 14, 2009

So after a few days of working on the matrices and its mathematical operations, theories and conceptional components (especially with matlab), I found a useful way to quickly determine whether a system is, either unique solution, no solution, or infinite solution.

First of all, I think I have thought too much about matrices and constructing a program. The objective, after spending a few days indeed, is probably just to find out "what number will produce" each of the three conditions. However, number goes on forever. But if we set a range for alpha and beta (the two variables we use) in the program, then there is a purpose for the program.

Matlab, just like any other programming language, has its own library, a collection of operations and pre-defined source-codes to use through the MatLab. For matrices, we do not need to worry about the operations. I probably had thought too much about "writing a program". I can still develop one, it isn't that hard once you know matrices' operations.

I can also make the program without doing the loop, simply just a program with the following sets of statements, and return a message to tell which condition is produce.

Now, the tools to determine each conditions quickly is to use the rule of "rank".

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
>> alpha = 3, beta = 4, A=[(alpha) 2 0;0 1 2; 0 0 0], b=[(beta);0;1], rref(A), X=A\b, rref([A b]), rank(A), rank([A b])

alpha =

     3


beta =

     4


A =

     3     2     0
     0     1     2
     0     0     0


b =

     4
     0
     1


ans =

    1.0000         0   -1.3333
         0    1.0000    2.0000
         0         0         0

Warning: Matrix is singular to working precision.

X =

   NaN
  -Inf
   Inf


ans =

    1.0000         0   -1.3333         0
         0    1.0000    2.0000         0
         0         0         0    1.0000


ans =

     2


ans =

     3

if rank(A) = rank([A b]) = u.s and infinite solution may exit
else
no solution exit

if rank(A) = # of unknowns, u.s exit
else
infinite solution exit.

If these statements are valid and true, then the first half of the program is just to create a loop and set a range to alpha and beta. The computer will print out the pair of alpha and beta when (1) there is no solution and (2) infinite solution. For unique solution, there is no need to print out the alpha and beta. Clear enough?

Filed under:CSC 1000, Matlab  Author:John Wong
No Comments »

working on matrices 3x3 in Matlab (1)

September 13, 2009

Okay, let's cut this short.

So far so good just need references to guide me through to crack down the concepts.

For Matrices, according to the alternative theorem,there should be 3 cases
1. Det =/ 0 and it's has unique solution
2. Det = 0, no solution
3. infinite solution

What I don't get is how do you determine something is infinite solution. To construct a program in Matlab isn't that difficult. All it takes is time to learn the ways to program in Matlab., The problem is the set of rules to determine the value of Alpha and Beta (input) and matrices (calculations). We can simply use det = or =/ 0 to determine something is either case 1 or case 2.

Well .... it seems something is working here.
when the last two numbers in the last rows are the same and the alpha and the first number in the last row are the same, it produces 0 det and infinite solution and also if we make the last row completely empty with zero, also infinite solution.
I guess I need to go through the references and see what I can do with it.

Filed under:CSC 1000, Matlab  Author:John Wong
No Comments »

SSH to Matlab in school

September 9, 2009

Mr. Yao was a very helpful person. He gave me this command to to test:

1
2
$ ssh -X 000.00.000.00 -l your_username
$ matlab

Instead, the -| didn't work out so I changed it around. Now this worked.

1
2
3
> ssh -X username@000.00.000.00
> password
> matlab

(more...)

Filed under:CSC 1000, Ubuntu  Author:John Wong
No Comments »

My first thought on Matlab

September 8, 2009

I will probably give another thought later when I get used to Matlab. As far as my study with the common programming languages, like Perl and C++, I do see a big difference in Matlab. This is more a mathematical language than what we usually expect from a programming language. However, people use Matlab for engineering projects because of the capacity of the mathematics offers by Matlab.

I don't know. I am the kind of person that needs real introduction to every language, from the most basic "Hello-World" script to the most advance (you write whatever you want!!!). I have to spend sometime finding a good book covering Matlab. There are some helpful sources online but the technical references they use, someone strike me down.

At the moment I am dealing with Matrices. Matrices aren't that difficult to understand, it's just math. But my pre-calc class was two years ago. I didn't remember any matrices operation at all. I probably need extra time to get use to the way Matlab writes program and how it works with numbers and command.

Here is the matrices

1
v=[1, 2, 3, 4], b=[4;5;6;7], v*b

Output

1
2
3
4
5
6
7
8
9
10
11
12
13
14
v =

1     2     3     4

b =

4
5
6
7

ans =

60

Filed under:CSC 1000, College Notes, Matlab, Programming  Author:John Wong
Comments Off

So, how's college?

September 5, 2009

CUNY started the new semester on August 28th, which was a Friday. My first class was music 10100 which started at 8:00am and last for the next 2 hours and a half, until 10:30am. Then I had my chemistry, FIQWS lecture and finally chemistry workshop.

August 28th was a week ago. So how's my college experience so far? Well, so far so good. I actually enjoyed being a college student. I am very satisfied with my chemistry professor, who called himself "a crazy professor". Since his first lecture (he brought his son to our first lecture), his demonstrations were always an epic win to the students. They scared the students (including myself, of course) though but we still loved to see them every in the end of every lecture! For example, he would put a handball and a banana into [i forgot the name of the liquid]. These objects lost all their heat and so they froze. Then, unexpectedly, he threw the handball across the lecture hall, and it broke with a sound just like breaking a glass. LMAO

He did so many amazing and great demonstrations which I hopefully will get a chance to record them (I have to ask his permission, of course). He did record his lecture because he used the "lecture text pad" to write notes and presented it on the screen. He recorded the pad and his lecture, but I doubted about the demo part.

Anyway. My calculus professor was a phD and he wrote textbook. Although our first two classes focused on high school materials, such as inequality and absolute values, he actually put effort into the presentations and used amazing examples to prove his points. I actually enjoyed the class. Yet, I hoped someone could turn down the air-conditioner, just way too low.

Anyway. I have an amazing time with CCNY. But as I progress, things get harder and more time consuming, I will eventually hate college. We have Monday off due to the national Labor Day, but I have a list of things to do, especially with the CCNY Student Group - indeed a pain in the neck.

Filed under:CSC 1000, Calculus 20100, Chemistry 10301, College Notes, FIQWS 10026, Music 10100, Personal  Author:John Wong
No Comments »