Deploy dabr on byethost

You may know that dabr requires the PHP module mcrypt to run, but byethost does not support it. so I modify the /dabr/common/user.php to bypass it:

function _user_encrypt_cookie() {
  $plain_text = $GLOBALS['user']['username'] . ':' . $GLOBALS['user']['password'] . ':' . $GLOBALS['user']['type'];
  /*
  $td = mcrypt_module_open('blowfish', '', 'cfb', '');
  $iv = mcrypt_create_iv(mcrypt_enc_get_iv_size($td), MCRYPT_RAND);
  mcrypt_generic_init($td, _user_encryption_key(), $iv);
  $crypt_text = mcrypt_generic($td, $plain_text);
  mcrypt_generic_deinit($td);
  return base64_encode($iv.$crypt_text);
  */
  return base64_encode($plain_text);
}
  
function _user_decrypt_cookie($crypt_text) {
  $plain_text = base64_decode($crypt_text);
  /*
  $td = mcrypt_module_open('blowfish', '', 'cfb', '');
  $ivsize = mcrypt_enc_get_iv_size($td);
  $iv = substr($crypt_text, 0, $ivsize);
  $crypt_text = substr($crypt_text, $ivsize);
  mcrypt_generic_init($td, _user_encryption_key(), $iv);
  $plain_text = mdecrypt_generic($td, $crypt_text);
  mcrypt_generic_deinit($td);
  
  list($GLOBALS['user']['username'], $GLOBALS['user']['password'], $GLOBALS['user']['type']) = explode(':', $plain_text);
  */
  list($GLOBALS['user']['username'], $GLOBALS['user']['password'], $GLOBALS['user']['type']) = explode(':', $plain_text);
}

That's it, very easy. But the risk is that your Twitter user name and password are saved as cookies in plain text.

Export your Twitter timeline to Google Calendar

This is the Python code to do the thing. I use Jython 2.5 and gdata 2.0.2. If you are running it behind proxy, you can set the environment as below:

set https_proxy=https://https_proxy
set http_proxy=http://http_proxy
set proxy-username=proxy-username
set proxy-password=proxy-password

And here is the source code:

try:
  from xml.etree import ElementTree
except ImportError:
  from elementtree import ElementTree
import gdata.calendar.service
import gdata.service
import atom.service
import gdata.calendar
import atom
import time
import datetime

gUsr = 'Gmail address'
gPwd = 'Gmail password'

tUsr   = 'Twitter user name'

# Depends on your tweets, 
# it should be 200/tweets
tPage  = 5
tCount = 200

# After run the script, 
# you can get the last id from the result
last_id = ''

twitURL = 'http://www.twitter.com/'

months_choices = {}
for i in range(1,13):
  months_choices[datetime.date(2009, i, 1).strftime('%b')] = i

class gCalendar:
  '''
  Refer to \gdata-2.0.2\samples\calendar\calendarExample.py
  '''

  def __init__(self, email, password):

    self.cal_client = gdata.calendar.service.CalendarService()
    self.cal_client.email = email
    self.cal_client.password = password
    self.cal_client.source = 'Twitter sync'
    self.cal_client.ProgrammaticLogin()

  def InsertEvent(self, title='title',
      content='content', where='where',
      start_time=None, end_time=None, recurrence_data=None):

    event = gdata.calendar.CalendarEventEntry()
    event.title = atom.Title(text=title)
    event.content = atom.Content(text=content)
    event.where.append(gdata.calendar.Where(value_string=where))

    if recurrence_data is not None:
      # Set a recurring event
      event.recurrence = gdata.calendar.Recurrence(text=recurrence_data)
    else:
      if start_time is None:
        # Use current time for the start_time and have the event last 1 hour
        start_time = time.strftime('%Y-%m-%dT%H:%M:%S.000Z', time.gmtime())
        end_time = time.strftime('%Y-%m-%dT%H:%M:%S.000Z',
            time.gmtime(time.time() + 1))
      event.when.append(gdata.calendar.When(start_time=start_time,
          end_time=end_time))

    new_event = self.cal_client.InsertEvent(event,
        '/calendar/feeds/default/private/full')

    return new_event

def userTimeline(tusr, count=5, page=1, since=1):

  statuses_user_timeline = twitURL + \
    'statuses/user_timeline.xml?screen_name=%s&count=%s&page=%s&since_id=%s'

  c = atom.http.ProxiedHttpClient()

  r = c.request('GET', statuses_user_timeline % (tusr, count, page, since))

  return r.read()

def feedCal(gService, xml):
  global last_id
  
  tree = ElementTree.fromstring(xml)

  status = tree.getiterator('status')

  for s in status:
    id = s.find('id').text
    c = s.find('text').text
    t = s.find('created_at').text
    t1 = t.split()
    t2 = [int(i) for i in t1[3].split(':')]
    st = datetime.datetime(int(t1[5]), int(months_choices[t1[1]]), int(t1[2]),
      t2[0], t2[1], t2[2])

    td = datetime.timedelta(seconds=1)
    et = st + td

    gService.InsertEvent(c, 'ID: %s' % id, 'twitter',
      st.strftime('%Y-%m-%dT%H:%M:%S.000Z'),
      et.strftime('%Y-%m-%dT%H:%M:%S.000Z'))

    # Get the first id as the last id
    if last_id < id: last_id = id

# Start here
def main():
  global last_id
  
  gCal = gCalendar(gUsr, gPwd)

  if last_id:
    feedCal(gCal, userTimeline(tUsr, tCount, 1, last_id))
  else:
    for i in range(1,tPage):
      feedCal(gCal, userTimeline(tUsr, tCount, str(i)))

  print '  Last id is', last_id

if __name__ == '__main__':
  print time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime()), 'start'
  main()
  print time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime()), 'complete'

Using Google Voice

Thanks to these two articles google.org.cn and wodingdong's blog, I finally got my GV number - (586) 330-9093, which is very similar to the number I'm using. After that, I tried to send sms to China(me, China mobile), Australia(haven't gotten feedback) and HongKong(three, 3G). Next, I think I will forward the GV to gizmo then to GTalk.

[TODO] A JavaScript Twitter client

I'm going to create a application that like the Gravity does on Symbian S60. This is to improve my HTML/CSS/JavaScript.

• This application is mainly written in JavaScript,
• Will be embedded to the Twitter proxy project birdnest, for some reason.

• Build UI/function for the personal timeline, as no login is needed.

Update on 17 Aug: First draft released on byethost. But only support IE and Safari now. Update on 14 Sep: Deploy it on the AppEngine, it's now support Firefox either.

Get something

This(4eZepuWr) would help me to get something that I want to have for a long time. I really want to post the English version here. But my English is poor. So I wait till now. Update: Finally, I can't get it.

Twitter

These days, I focused on the Twitter. I got many news/information from it. But thanks to the firewall, I know even more on the proxy.

Access Kaixin001.com via Jython

Some notes:

1. Use cookies in the request
2. Use tools(webscarab) to get the request URL/data in the Flash
3. Parse the HTML/XML

Here is the update Proxy.py(base on the proxyHTTP.py in the last post) to build proxy under Jython:

from java.net import Authenticator
from java.net import PasswordAuthentication
from java.lang import System
from java.net import URL
from urllib2 import ProxyHandler, build_opener, HTTPCookieProcessor

proxy_info = {
'user' : '*domain*\\*user*',
'pass' : '*password*',
'host' : '***',
'port' : '***'
}

class auth(Authenticator):
    def getPasswordAuthentication(self):
        return PasswordAuthentication(proxy_info['user'], proxy_info['pass'])

def getHTTPS(url):
    systemSettings = System.getProperties()
    systemSettings.put("https.proxyHost", proxy_info['host'])
    systemSettings.put("https.proxyPort", proxy_info['port'])
    Authenticator.setDefault(auth())
 
    u = URL(url)
    d = u.openConnection().getInputStream()
    c = d.read()
    r = []
    while c <> -1:
        r.append(chr(c))
        c = d.read()

    return ''.join(r)

def HTTPProxyHandler():
    return ProxyHandler({'http' : \
        'http://%(user)s:%(pass)s@%(host)s:%(port)s' % proxy_info})

def getOpener(cj):
    # cj - use cookie or not
    proxy = HTTPProxyHandler()
    if cj:
        opener = build_opener(proxy, 
                              HTTPCookieProcessor())
    else:
        opener = build_opener(proxy)
        
    return opener
    
def main():
    urlpath = r'https://www.google.com/reader'
    print getHTTPS(urlpath)

if __name__ == '__main__':
    main()

And this is the main script:

import Proxy
import urllib2
#import cookielib
import re
from urllib import urlencode
from xml.dom import minidom
from time import sleep
from random import random


urlparm = {
    'garden': '/!house/!garden//getconf.php',
    'water': '/!house/!garden/water.php',
    'antigrass': '/!house/!garden/antigrass.php',
    'antivermin': '/!house/!garden/antivermin.php',
    'havest': '/!house/!garden/havest.php',
    'plough': '/!house/!garden/plough.php',
    'ranch': '/!house/!ranch//getconf.php',
    'r_havest': '/!house/!ranch/havest.php'
}

xmlparm = '%s'

class kx:
    def __init__(self):
        self.buildProxy()
        self.url = 'http://www.kaixin001.com'
        self.DELAY = 3  #3 sec
        
    def buildProxy(self):
        # build a new opener that uses cookies
        #proxy_support = Proxy.HTTPProxyHandler()
        #opener = urllib2.build_opener(proxy_support, 
        #                            urllib2.HTTPCookieProcessor())
        #opener.addheaders = [('User-Agent','Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)')]
        opener = Proxy.getOpener(True)
        
        # install it
        urllib2.install_opener(opener)
        
    def login(self, usr, pwd):
        
        path = self.url + '/login/login.php'
        
        body = {
            'url': '/home/',
            'email': usr,
            'password': pwd
        }
        
        req = urllib2.Request(path, urlencode(body))
        
        #response = urllib2.urlopen(req).read()
        #print response
        urllib2.urlopen(req)        
        
    def getpage(self, path, data={}):
        #Delay
        #sleep(self.DELAY+random()+random())
        
        req = urllib2.Request(self.url+path, urlencode(data))
        
        return urllib2.urlopen(req).read()
        

def get_friend_info(str1, friend_info):
    pattern = '\?uid\=(\d{6,8})\"\stitle\=\"([^\"]+)\"'
    for match in re.finditer(pattern, str1):
        friend_info[match.group(1)] = unicode(match.group(2),'utf-8')

def get_verify(str1):
    pattern = 'g_verify\s=\s\"([^\"]+)\"'
    return re.findall(pattern, str1)[0]

def showMsg(fuid, t):
    if fuid=='0':
        print '%s: %s' % (t, 'My')
    else:
        print ' %s: %s' % (t, fuid)

def pxml(t, c):
    try:
        xmldoc = minidom.parseString(xmlparm % c)
    except:
        print '! xml parse failed.'
        return
        
    if xmldoc.getElementsByTagName('ret')[0].firstChild.data == 'fail':
        print '  %s: %s' % (t, xmldoc.getElementsByTagName('reason')[0].firstChild.data)
    else:
        print '  %s: %s' % (t, 'succ')

def showFriend(o, v):
    c = o.getpage('/friend/')
    i = {}
    get_friend_info(c, i)
    for k in i:
        if k in ['*ignore list*']: continue
        #print k, i[k]
        showGarden(o, v, k)
        showRanch(o, v, k)
    #print '-'*20
    #print 'Total %d friends' % len(i)
    
def showGarden(o, v, fuid):
    
    pars = {
        'fuid': fuid,
        'verify': v
    }
    
    c = o.getpage( urlparm['garden'], pars )

    try:
        xmldoc = minidom.parseString(xmlparm % c)
    except:
        print '! Failed to parse %s xml file.' % fuid
        return

    showMsg(fuid, 'Garden')
    
    for nodes in xmldoc.getElementsByTagName('garden'):
        for node in nodes.getElementsByTagName('item'):
            if node.getElementsByTagName('name'):
                
                farmnum = node.getElementsByTagName('farmnum')[0].firstChild.data
                water = node.getElementsByTagName('water')[0].firstChild.data
                grass = node.getElementsByTagName('grass')[0].firstChild.data
                vermin = node.getElementsByTagName('vermin')[0].firstChild.data
                cropsstatus = node.getElementsByTagName('cropsstatus')[0].firstChild.data
                shared = node.getElementsByTagName('shared')[0].firstChild.data
                grow = node.getElementsByTagName('grow')[0].firstChild.data
                totalgrow = node.getElementsByTagName('totalgrow')[0].firstChild.data
                
                pars = {
                    'fuid': fuid,
                    'farmnum': farmnum,
                    'seedid': 0,
                    'verify': v,
                    'r': random()
                }
                #print node.getElementsByTagName('name')[0].firstChild.data
                
                if int(water) < 5:
                    pxml( 'Water', o.getpage( urlparm['water'], pars ) )
                
                elif grass == 1:
                    pxml( 'Grass', o.getpage( urlparm['antigrass'], pars ) )
                
                elif vermin == '1':
                    pxml( 'Vermin', o.getpage( urlparm['antivermin'], pars ) )
                
                # me
                elif fuid == '0' and grow <> '' and grow == totalgrow and \
                    ( cropsstatus == '1' or cropsstatus == '2' ):
                    
                    pxml( 'Havest', o.getpage( urlparm['havest'], pars ) )
                    
                    if shared == '0':
                        pxml( 'Plough', o.getpage( urlparm['plough'], pars ) )
                
                # others
                elif fuid <> '0':
                    #  plough the shared item only
                    if cropsstatus == '3' and shared == '1':
                        pxml( 'Plough', o.getpage( urlparm['plough'], pars ) )
                    
                    # steal
                    elif cropsstatus == '2' and shared == '0':
                        
                        bsteal = True
                        
                        for nc in node.getElementsByTagName('crops')[0].childNodes:
                            if nc.data.find('font ') <> -1:
                                bsteal = False
                                break
                            
                        if bsteal:
                            pxml( 'Steal', o.getpage( urlparm['havest'], pars ) )
                    

def showRanch(o, v, fuid):
    
    pars = {
        'fuid': fuid,
        'verify': v
    }
    
    c = o.getpage( urlparm['ranch'], pars )
    
    try:
        xmldoc = minidom.parseString(xmlparm % c)
    except:
        print '! Failed to parse %s xml file.' % fuid
        return
    
    showMsg(fuid, 'Ranch')
    
    for nodes in xmldoc.getElementsByTagName('product2'):
        for node in nodes.getElementsByTagName('item'):
            
            skey = node.getElementsByTagName('skey')[0].firstChild.data
            typenum = node.getElementsByTagName('type')[0].firstChild.data
            
            pars = {
                'skey': skey,
                'fuid': fuid,
                'foodnum': 1,
                'type': typenum,
                'seedid': 0,
                'verify': v,
                'r': random()
            }
            
            bsteal = True
            
            if fuid == '0':
                pxml( 'Havest %s' % skey, o.getpage( urlparm['r_havest'], pars ) )
            else:
                for nc in node.getElementsByTagName('tips')[0].childNodes:
                    if nc.data.find('font ') <> -1:
                        bsteal = False
                        break
                    
                if bsteal:
                    pxml( 'Steal %s' % skey, o.getpage( urlparm['r_havest'], pars ) )
                    

def main():
    usr = [
        ('*email*','*password*')
    ]
    
    for u,p in usr:
        print '-'*18, u, '-'*20
        o = kx()
        
        o.login(u,p)
        
        c = o.getpage('/!house/garden/index.php')
        v = get_verify(c)
        
        showGarden(o, v, '0')
        showRanch(o, v, '0')
        
        #print '-'*18, 'Friends', '-'*20
        showFriend(o, v)
        

if __name__ == '__main__':
    main()

Update Google App Engine with Jython

Here is the background/software: JRE 1.5.0_15, Jython 2.5b3, AppEngine 1.2.0 and a authentication proxy for connecting to Internet.

Due to the issue of the HTTPS proxy handler in Python library urllib2, I can't use Jython to udpate my app to AppEngine. After searhed the web I came up to this solution - use Java class to get html from HTTPS, but still use urllib2 for HTTP proxy.

1. Edit the file google-appengine\google\appengine\tools\appengine_rpc.py, add the line in the header.

import ProxyHTTP

Then in the line 178, update as

      response_body = ProxyHTTP.getHTTPS(req.get_full_url() + '?' + req.get_data())
      #response = self.opener.open(req)
      #response_body = response.read()

And line 359, update as

    #opener.add_handler(urllib2.ProxyHandler())
    opener.add_handler(ProxyHTTP.HTTPProxyHandler())

2. Create the file google-appengine\google\appengine\tools\ProxyHTTP.py

from java.net import Authenticator
from java.net import PasswordAuthentication
from java.lang import System
from java.net import URL
from urllib2 import ProxyHandler

proxy_info = {
'user' : 'domain\\user',
'pass' : 'password',
'host' : 'host',
'port' : 'port'
}

class auth(Authenticator):
    def getPasswordAuthentication(self):
        return PasswordAuthentication(proxy_info['user'], proxy_info['pass'])

def getHTTPS(url):
    systemSettings = System.getProperties()
    systemSettings.put("https.proxyHost", proxy_info['host'])
    systemSettings.put("https.proxyPort", proxy_info['port'])
    Authenticator.setDefault(auth())
 
    u = URL(url)
    d = u.openConnection().getInputStream()
    c = d.read()
    r = []
    while c <> -1:
        r.append(chr(c))
        c = d.read()

    return ''.join(r)

def HTTPProxyHandler():
    return ProxyHandler({'http' : \
        'http://%(user)s:%(pass)s@%(host)s:%(port)s' % proxy_info})


def main():
    urlpath = r'https://www.gmail.com'
    print getHTTPS(urlpath)

if __name__ == '__main__':
    main()

That's all. But there are still some issues when using Jython for AppEngine. Such as:

• Before upload files, there is a warning: "Problem with getpass. Passwords may be echoed".
• When run the app locally, there is no PIL(I did not install it as it might have many problems).
• When run the app locally with google framework, there is a ImportError: No module named operator.

My friend came back from HK

He came back in the Easter but will go to visit his ancestor's grave. He lived here before he graduated from university. Then he settled down in HK.

Automation on L-Note

Refer to this page: http://www-12.lotus.com/ldd/doc/lotusscript/lotusscript.nsf/ 1efb1287fc7c27388525642e0074f2b6/1f82ab14864680138525642e007687cb?OpenDocument

I want to create a JS script to list all the mails in the Inbox. As I can't use com object in Python on my workstation. But it failed to get value of "from" while VBS can do the job. I don't have enough time to investigate the root cause, just leave it here for later reference.

'VB script

Set objNotesSession = CreateObject("Lotus.NotesSession")
objNotesSession.Initialize(pwd)
Set objNotesMailFile = objNotesSession.GetDatabase(server, database file)
Set view = objNotesMailFile.GetView("($Inbox)")
Set doc = view.GetFirstDocument
While Not ( doc Is Nothing )
  WScript.Echo doc.GetItemValue( "From" )(0)
  Set doc = view.GetNextDocument( doc )
Wend

//JavaScript

var s = new ActiveXObject("Lotus.NotesSession");
s.Initialize(pwd);
var db = s.GetDatabase(server, database file);
var v = db.GetView("($Inbox)")
var doc = v.GetFirstDocument();
while (doc) {
  WScript.Echo( doc.LastAccessed );
  doc = v.GetNextDocument(doc);
}