NEETHACK!

Live number in ruby

Live number is a number of the total sum for birthday year,months and days.
Below is the ruby implementation:

def live_number(birthday)
  reduce("#{birthday.year}#{birthday.month}#{birthday.day}")
end

def reduce(sum)
  add_sum = each_digits(sum).reduce(:+)
  add_sum >= 10 ? reduce(add_sum) : add_sum
end

def each_digits(num)
  num.to_s.split("").map{|c| c.to_i }
end

spec:

describe "live_number" do 

  before :each do 
    @birthdays = [Time.new(2001,3,5),
                  Time.new(1978,5,26),
                  Time.new(1962,10,1)]
    @birth = {:day => Time.new(1981,6,11), :ans => 9 }
  end

  it "return number of live by birthday" do 
    @birthdays.each do |day|
      live_number(day).should be_a_kind_of(Fixnum)
    end
  end

  it "return the correct number of live" do
    @birthdays.each do |day|
      live_number(day).should == 2
    end
  end

  it "return correct number with specific birthday" do 
    live_number(@birth[:day]).should == @birth[:ans]
  end
end

Ruby Warrior!!

Ruby Warrior is a game that teach you how to programming with Ruby, and also how to write AI script.

Install

gem install rubywarrior
rubywarrior

And you will have a player.rb script and README,

sample code for player.rb

def play_turn(warrior)
    if warrior.feel.empty?     
      warrior.attack!
    elsif warrior.health <= 5
      warrior.rest!
    else
      warrior.walk!
    end
  end

follow the instruction to implement player.rb, then run the rubywarrior to start fighting with your warrior!!

Termkit - Graphical Terminal

Termkit is an interesting project that focus on the improvement of terminal.

The UNIX terminal we use right now is established in the time of mainframe.

It is still powerful because it's simplicity and ability to script. The terminal output can be pipelined as input to other programs, and the command can be easily scripted and recognized by programs. Unlike GUI, it's hard to script the simple action like: click this red button and open this file. However, because the command is used by both human and machine. It is pretty unfriendly for beginner and unfriendly in terms of today's UI design. The target of termkit is adding a visual layer for terminal. like we can output the command in different color, the folder and files type with icon, autocomplete with commands and tooltip of command switches.

The structure is like this:

input => process => output for program
||
termkit => output for human

and termkit is using webkit for visual presentation, and node.js for glue the process with webkit.

Install:

install node.js , npm

npm install mime (for webkit to distinguish content type)

git clone https://github.com/unconed/TermKit.git --recursive

run the termkit demean

cd TermKit/Node; node nodekit.js

run client

cd Termkit/Build; unzip Termkit.zip; open Termkit.app

This project is still on an early stage. But it show an different aspect of GUI improvement. Like an different approach of sikuli.
https://github.com/unconed/TermKit

Introduction of SCSS and Compass

Recently, the CSS tool like SCSS,SASS and less have become more popular.
Those framework provide several extra functionality to CSS, also with build in compress option.
take SCSS as example:

install scss using ruby gem:

gem install sass
    sass style.scss style.css 
    sass [INPUT] [OUTPUT]

functions

1. Nesting class

#css{
     padding:5px;
     .tool{
          padding-top:10px;
     }
     //parent reference
     &:hover{
          background-color:#FFF;
     }
}

2.Variables
you can use dollar sign as variable mark in SCSS:

$mid-padding : 5px;

#css{
     padding : $mid-padding;
}

3. Mixins
Mixins means module inheritance in Ruby. It's just like inheritance from abstract class in SCSS. Use @ sign for declare maxin and include in css class.

@mixin block-base($margin){
     margin:$margin;
     padding:$padding;
}

#block{
     @include block-base(10px);
}

4. Inheritance
Also you can use @extend to inheritance from other CSS class.

.error{
     color:#F43;
}

.big-error{
     @extend .error
     font-size: 22px;
}

5. import
Import can let you include library and mixins into current css file, like compass will have lots of library to import.

@import "compass/css3/border-radius";
@import "compass/utilities/general/clearfix";

Compass

Compass is another framework that integrate SCSS and CSS grid framework like blueprint,
that provide you build in CSS layout, ie hack and other support. Compass is well integrated with rails.

using compass:

gem install compass
    compass init (under rails directory)

will create directory in app/stylesheets

compass compile

can compile into public/stylesheets directory

more reference for compass library

Backbone.js - MVC framework for front-end javascript

What is Backbone.js?

Backbone is a framework for front-end JavaScript, unlike jQuery focus on easier DOM manipulation and event binding, backbone provide a structure for separating data model and DOM, just like a MVC framework separate model, view and controller. Make heavy JavaScript application easier to develop and maintain.

Why Backbone?

In jQuery, we will write a sequence of chain selector for assign data model to DOM element like:

var article = {
  author:"Joe",
  content: "testing"
};
$('#article').click(function(event){
  $(this).find('.content').text(article.content);
});

For backbone, it provide the Model class and View class, which is the main two element of backbone.

var Article = Backbone.Model;

var article = new Article({
  author:"Joe",
  content: "testing"
});

var ArticleView = Backbone.View.extend({
  el: $('#article'),
  initialize: function(){
    _.bindAll(this,'updateContent');
    this.model.bind('change',this.updateContent);
  },
  events: {
    "click .content" : "updateContent"
  },
  updateContent: function(){
    this.$('.content').text(this.model.get("content"));
  }
});

var articleView = new ArticleView({model:article});

OK, it's seen like a completely overwork, that's my first thought when I see a clientside MVC framework too, but we have to dig further to understand why people create it.

Event Binding

First, it separate the data model and view - which include event handle and DOM element, we can easily bind data model in view like:

this.model.bind('change',this.updateContent);

It means when the data model change, the view will be automatically updated to DOM element.

Also, backbone provide literal event binding:

events:{
  "click .content": "updateContent"
}

which bind the "click" event to ".content" class, trigger "updateContent" function in current view.
It's useful when binding multiple events.

Restful synchronization

Furthermore, the backbone provide a RESTful format API for synchronize data model with server.
every model class in backbone have a corresponding url for synchronize, and when the "model.save" or "model.fetch" method is called, the model will make an Ajax request to update data with server.
For example:

article.save({author:"John"});

will first find article.sync method, if undefined, call Backbone.sync.
The sync method will make an ajax request to server, and update model, trigger event and update to view.

the url of model is defined by RESTful API:

create -> POST /model
read -> GET /model[/id]
update -> PUT /model/id
delete -> DELETE /model/id

and we can also change it by overwrite url parameter

Dependency

backbone.js depend on underscore.js and jQuery/Zepto library, since it is only a 3.9k framework focus on the structure of JavaScript application.

Collection and Controller

backbone provide Collection, which is the collection of Backbone.Model,
the Controller in backbone is more like the route map in MVC framework, it provide clientside fragment url routing, eg. "/#article/12" match "article(id)". So the url can be recorded and bookmarked by browser.

Template

In Backbone, we can create template in view.
In that way we can add and remove multiple view and render with template.
The underscore.js provide template engine or we can use other engine too.

In view:

render:function(){
    var template = "<%=content %>";
    $(this.el).html(
        this.template(this.model.toJSON())
    );
    return this;
}

Conclusion

JavaScript MVC framework is good for rich client application and also can keep your javascript cleaner.
But whether to use it depend on how much script in your application, for small page, the MVC is total overhead, but it is good for large scale page like Mail or Desktop like application.

Backbone
Sample todo application
Annotated source

[Note] Start first instance on Amazon Web Service EC2

How-to guide by Amazon
http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/

Amazon have become the largest cloud service company that provide compute instance and stroage
in their data center, that we can rent those computing resource as instances.

What's EC2?
Elastic Compute Cloud, the amazon computing service, we can rent an instance and run service on it as a virtual machine.

Amazon also have other service like S3 - simple storage service , is a storage service that you can upload file and let other people download. S3 provide auto bitTorrent support for big files.

Start an Instance

Login Amazon Aws console , choose EC2 tag, press "launch instance button"

1. Choose Instance Type: AMI(amazon machine image)
Default has Amazon custom Linux, Suse Linux and Windows Server 2003, both 32bit and 64bit.
There's also community AMI you can choose (which have CentOS and Ubuntu)

Read ubuntu document to find official AMI for ubuntu: https://help.ubuntu.com/community/EC2StartersGuide

ami-06ad526f for free micro instance supported Ubuntu 11.04 , check the Star mark for free support or not.
The root device ebs means the data will be saved in ebs service by default, which will not disappear after shutdown,

2. Select number of instance, instance type (related to money, micro is free)

3. Advance Instance Option: kernel, ramdisk id, monitoring, import user data shutdown behavior
you can import your userdata as text or file, that's useful when setting up multiple instances, you don't have to set user separately.

4.key-value tag to administrate multiple instances, and help to manage
ex: Owner = Jimmy
Admin = Jimmy
Service = Redmine
Stack = Development

5.choose key-pair to login with ssh , pem format (see below for how to login)

6.choose security group
Decided which incoming network traffic should be delivered to your instance
- access web traffic on port 80
- Its like the firewall setting, need to add every open port for usage (black-listed)
- SSH:22, windows RDP(Remote Desktop Protocol)
- source : 0.0.0.0/0 means no restriction on ip/submask
- 192.168.2.0/24 means restriction on 192.168.2 sub-area

How to login?

pem: public key of a specific certificate. Apache use this kind of certificate.
In apache installs, this frequently resides in /etc/ssl/servercerts.
it is also called X.509 Certificates

terminal:
# ssh -i ~/.ssh/XXX.pem root@ec2-##-##-##-##.compute-1.amazonaws.com

connection in windows using Putty:
generate private key from .pem file using puttygen,
choose Load => XXX.pem
then save private key XXX.ppk

Connection with putty under windows
choose connection => auth => load private key

More on aws:

EC2 command line tool: using command line to manage EC2 Instance
need to apply X.509 keypair for Amazon EC2 AMI Tools,
Amazon Web Services > Your Account > Security Credentials > X.509 Certificate

Watchr : simple continuous testing tool

Watchr is a pretty cool and simple tool for saving developer's time.

According to the preject description, watchr is:

Agile development tool that monitors a directory tree, and triggers a user defined action whenever an observed file is modified. Its most typical use is continuous testing, and as such it is a more flexible alternative to autotest.

But moreover, Watchr can be used on every simple task that need to execute a file, like markdown generate, coffee script compile, and the syntax is so simple to add those task in watchr script.

The syntax is like:

watch 'regular expression' { |matched_file| action_for matched_file }

#listen to all file and print filename when it modified
watch('.*') {|match| puts match[1]}

#listen to ruby file and run test
watch('lib/(.*)\.rb') {|match| system("ruby test/test_#{match[1]}.rb") }

#match[0] is full filename, match[1] is filename without extname.

running under command:

watchr script_name

Continous Testing

Continous Testing is a concept born from MIT Program Analysis Group,
It improve the Test Driven Development, which is auto run the corresponding test
in TDD right after the code saved. In this way, the test time lag will be reduced to zero,
and programer won't need to run the test to know the program is work or not. Makes testing become natural behavior in development.

Refers

ihower's rails3 watchr script template
Doc generate template from watchr project itself (dog fooding)

mini BDD Framework in CoffeeScript

Rails 3.1 is default supporting Coffee-Script,

This might be controversial, but basically I think it is a right move,
since rails development philosophy is always simplify developer's life.
And coffee script sure is.

compare the following code with javascript and coffee script:

describe('spec in javascript', function(){
  it('should return true in javascript',function(){
    return 1===1;
  });
});

//in coffee script
describe 'spec in coffeescript', ->
  it 'should return true in coffeescript',->
    1 is 1

You can see how the syntax has been simplified and easy to read.

Inspired by A unit test framework in 44 lines of ruby, I want to write a test framework in coffeescript to show the good of it.

Here's the work:

dsl =
  tests : {}
  it : (description,test) ->
    @tests[description] = test
  parse:(block)->
    block.call(dsl)
    @tests

class Executor
  constructor: (@description,@tests) ->
  tests : {}
  success : 0
  fail : 0
  count : 0
  run:()->
    console.log(@description + " ::")
    @execute desc,test for desc,test of @tests
    console.log("Executed #{@count} tests: #{@success} success, #{@fail} fail")

  execute : (desc,test) ->
    test_result = test()
    if test_result is yes than @success += 1 else @fail += 1
    @count += 1
    console.log(" --"+ desc + " is "+ test_result)

exports.describe = (description,block) ->
  executor = new Executor(description,dsl.parse(block))
  executor.run()

with the following testcase

spec = require './nspec'

spec.describe 'hello coffee test', ->

  @it 'should be true', ->
    1 is 1

  @it 'should fail', ->
    1 is 2

Basically the describe function contains all tests in callback,
and will parse all the testcase by

block.call(dsl)

which inject the dsl functions as the "this" object in block, and the "it" syntax will collect the test functions to a map object.
afterward, Executor will execute all tests and print out results.

During the implementation, I can't find the way to remove that nasty @ before test function 'it'
without declaring global. I am still working on it, there's a solution that you can execute test
by calling node.js child_process "coffee spec.coffee"(from coffee-spec), but that will be too complex for this. Jasmine didn't have this problem, let me know if anyone got the answer, or I will figure it out later :p.

Foursquare API with oauth2 gem

The oauth things is pretty complex, here is some notes that may avoid you fall into the same pitfall as me.

You can read the whole spec here.

But basically, it provide a interface let website can access the user's data in other website.
User will first be redirect to that website, click confirm, and redirected back to the callback page.

In the back end, the process of server is:
1.Send request to target website for token, with the server's client_id & secret_id
2.Redirect user to authorize page with token (and callback url)
3.After user confirm, user will be redirect to the callback page with a access_token
4.Save the access_token, and free to call target server for user data.

However, using gem to manage those things seen like a smart way, but still, it may not work correctly because of some error:

#pitfall 1

Check the api is using oauth or oauth2,
which is different when choosing the oauth gem.

#pitfall 2

check the oauth_path, access_token_path is correct
the default setting of gem may not work with the api you choose.

#pitfall 3

check the api address,
in foursquare, the authorize url is : https://foursquare.com/oauth2/authorize
however, the api is not under the same subdomain, is under https://api.foursquare.com/

make sure those setting is correct, and you will feel the sweet of oauth!

here's the code with sinatra:

require 'sinatra'
require 'oauth2'

get '/auth' do
  redirect to(client.web_server.authorize_url(:redirect_uri =>
                                              "http://myhost/callback"))
end

get '/callback' do
  #make sure the access token is named "code"
  access_token = client.web_server.get_access_token(params[:code],
                                      :redirect_uri => "http://myhost/callback")
  access_token.get('https://api.foursquare.com/v2/users/self')
  # save your access token in session or db...
end

def client
  client ||= OAuth2::Client.new(
  YOUR_CLIENT_ID,YOUR_SECRET_ID,
  :site => 'https://foursquare.com',
  :authorize_path => '/oauth2/authorize',
  :access_token_path => '/oauth2/access_token')
end

HTML5 push state

Recently I was trying to make a slide pages effect, and I realize there are some obstacle to let the browser work properly on next/back page. because you can only change the #hash value of url but the real url without redirect, the old ajax problem.

But wait a minute, is the github use the ajax page slide effect and work perfectly? After a peek on the source, I found they are using the html5 pushstate api on webkit and ff4, which can let you change the url without actually redirect the page.

sample:

$(function(){
  //check if the browser support it.
  $.get('/new_page',function(data){
    if(history.pushState){
      //push the url to the browser history, with state object and title.
      history.pushState({isAjax:true},"push the state"), 'new_page');
    }
    //update the data...
  });
  //popstate event will triggered whenever the url changed
  //but the page won't reload if url is pushed into history
  //so you have to update page yourself or reload it.
  $(window).bind("popstate",function(e){
    if (e.state.isAjax) {
      //make the ajax call for update page
    }
  });
});

try it yourself by typing

history.pushState({},"test","/test:);

in your browser console, and you'll see the effect.

reference: https://developer.mozilla.org/en/DOM/Manipulating_the_browser_history

[notes]Macvim settings

1.remove ^M (Ctrl-V Ctrl-M, dos newline symbol) when vimrc file is created in Windows:

search and replace in vim commandline by typing:

:%s/\r//g

to replace the ^M space

or using tr command to delete ^M char in file:

cat .vimrc | tr -d '\r' > .new_vimrc

2.open macvim from terminal:

add application path to .bash_profile

ex:
export PATH="$PATH:/Applications"

call mvim script from terminal to open macvim

3.remove toolbar and toolmenu:

set guioptions-=T
set guioptions-=m

4.sample .vimrc from vgod

https://github.com/vgod/vimrc/blob/master/vimrc

F2E Interview Questions

Here are some interview questions from a front-end engineering job, through I didn't get that position , these questions are pretty interesting and might help the others who want to get a Front-end job. So here are the questions:

1.Weird Width:

I create this <div/> to have my article inside. However, my nit picking product manager finds the width of this block is a little different between Internet Explorer 6 and Firefox. Can you help me to fix this weird bug?
In Internet Explorer, the width is 500px.
In Firefox 3, the width is 522px.
Can you help to correct my attached file so that the page is identical in different browsers? It would be great if you can tell me why so that I won't do that stupid thing again. And if you have spare time, please style the page for me since I am so lazy. ^^』』

Ans:

First of all, the problem of weird width is because older IE browser has different explanation in CSS box model.

The IE explained CSS width attribute as the width of content + padding + border. As the result, we get the width of 500px in IE browser that includes padding and border, in contrast of 522px in other modern browser.

Internet explorer 6+ has two mode of CSS interpretation:
Quirk mode – the interpretation of IE's own standard
standard-compliant mode – according the W3C standard (almost)
By define the <!DOCTYPE> tag we can switch the IE to standard-compliant mode. So here's the solution:
<!DOCTYPE>
By attach this before <html> tag, we can get correct width

2.Horizontal Toolbar

Product manager asks me to make a horizontal toolbar for our website.
Even if user scrolls, toolbar will always stay at the bottom of page. Another requirement is that all modern browsers are compatible with this new feature. (Internet Explorer 6 - 8, Firefox, Opera, Google Chrome, and Safari) After making a prototype I find it's really difficult. Please teach me how to do that.

Ans:

The CSS { position: fixed } attribute can build the toolbar in same position whenever scrolling. So the solution is:
#toolbar { position: fixed ; bottom:0; width:100%; }
Unfortunately, our old IE6 friend doesn't support it.
Again, search for some hack at web, we found 2 approaches:
1. JavaScript approach: which auto re-assigns position when event called.
2. CSS approach:
Using CSS expression or, Using wrapper
I use wrapper method because it's the simplest to me.

Normally, the { position:absolute } attribute will anchor the element according root element. Therefore we can set another element outside to be the root and have height equal to the screen. So the { position:absolute } attribute can place toolbar in right position. And by set the {overflow:hidden} attribute, we can hide the content outside the root element.
When assign css attribute, IE will active the element. So we can active the <html> element to be the wrapper, <body> to be the content,
CSS example:

#toolbar {position: fixed; bottom: 0; width: 100%;}
//css for ie6
* html{ overflow-y: hidden; } //hide the wrapper scroll
* html body{
  padding:0px;margin:0px; // reset
  overflow-y: auto; //set scroll
  height: 100%;
}
* html div#toolbar
{
  position: absolute;
  width:98%; //fix the problem that toolbar over scroll
}

Show me the Browser

Dear Candidate,
We front-end engineers are always busy at switching between browsers. Each engineer has three browsers installed on their notebook. They are Internet Explorer 6, Internet Explorer 7, and Firefox 3. It's really confusing to identify which browser is using. As an engineer with some repute, you want facilitate others by showing icon.

Ans:

1. we need to set CSS sprite for browser picture
I try to build CSS class with background-position attribute:
.firefox{ background-position:0 0 }
.ie7{ background-position:0 -75px }
.ie6{ background-position:0 -150px}
2. JavaScript to detect browser and parse userAgent string and replace picture for right browser.
using library from here: http://www.quirksmode.org/js/detect.html

window.onload = function(){
  // get the DOM object by ID
  var browserPic = document.getElementById("browsers");
  //detect browser and set correct class
  if(BrowserDetect.browser == "Firefox"){
    browserPic.className = "firefox";
  }
  if(BrowserDetect.browser == "Explorer"){
    if(BrowserDetect.version == "6")
      browserPic.className = "ie6";
    if(BrowserDetect.version == "7")
      browserPic.className = "ie7";
  }
};

(but it turn outs that using css hack * for ie6, _ for firefox is a far more simpler solution)

Scheme Note

Recently I am playing with some scheme code, here's some introduction and code snippet about it.

What is Scheme?

Scheme is a pure functional programming language that was developed by MIT media lab.
The design philosophy of scheme is, rather than provide more features, it's more important to provide elegant core functions, minimize the weak points and restriction of the languages.(which is: less is more)

A functional programming language, as the name, is a language that treat function as first level primitive. In functional language, you can pass and return function as variable, the program in functional language is constructed by multiple recursive function call.

for example, a for loop iterate a data array in structure language is like:

int[] a = [0,1,2,3,4]
for (int i = 0 ; i < 5 ; i++){
  print(a[i]);
}
// 01234

in scheme you have to define a for-each loop like this and pass the function:

(define each (lambda (list proc)
  (if (not (null? list))
      (begin
        (proc (car list)) ; car : get first element of list
        (each (cdr list) proc) ; cdr : get the rest of list except the first element
      )
)))

(each '(0 1 2 3 4) write) ; pass the function "write" to execute on each element
; 01234

In functional language, a function is more like a logic unit, take inputs and return output, it's more close to the function in Mathematic. Therefore, a function will not have any side effects and dependencies, which make it very easy to test and reuse.
Another good part in functional language is lazy-evaluation. The function passed will only executed when it's needed to, for example, a program in structure programming like:

print(db.read());

Will call the read() function and save in memory before print, but in functional language, the function call is strict synchronization:

(print (read db ))

The read function will only perform when print function is going to print, and only read the data that is needed to print, if the print function abort, the read will not be executed. In this way, the function call can be integrated and optimized. Also, because the only things effect function output is the input, so we can easily save the results of function and reuse it. In Scheme, you can call the "delay" function to cache the results of function output: http://www.ibm.com/developerworks/linux/library/l-lazyprog.html

functional language is mainly used on science, math ,AI and financial area,

the functional language on the block included: scheme, haskell, f#, clojure and more.

more information about fp:
why fp matter
why fp matter matter

Getting Started

Racket is a good implementation and IDE for scheme, it's more friendly then MIT-Scheme, you can install it and choose the language as "R5RS" which is the newest version of scheme implementation.

snippets

get last element or list:

(define last (lambda (list) 
             (if (> (length list) 1) 
                 (rac(cdr list)) 
                 (car list) 
             )))

fibonacci:

(define fib (lambda (level)
            (if (< level 2) 
                 level
                 (+ (fib(- level 1)) (fib (- level 2)) ) 
            )))
; dynamic programming version
(define (fib-dym level) 
        (let fib ((base0 0) (base1 1) (n level))
        (if (< n 1)
            base0
            (fib base1 (+ base0 base1) (- n 1)))))

quick sort:

(define (partition pivot num-list)
  (if (null? num-list) '(()()) 
      (let ((split-of-rest (partition pivot (cdr num-list)))) ;split of rest is a 2 way list:((first...) ( second...))
        (if (< (car num-list) pivot)
            (list (cons (car num-list) (car split-of-rest)) ;put x into first list
                  (cadr split-of-rest))
            (list (car split-of-rest) ;put x into second list
                  (cons (car num-list)(car (cdr split-of-rest))))
            ))))

(define (quicksort num-list)
  (if (<= (length num-list) 1) num-list
      (let ((split (partition (car num-list) (cdr num-list))))
        (append (quicksort (car split)) ;smaller
                (list (car num-list))   ;pivot
                (quicksort (cadr split) ;larger
                           )))))

simple test function:

(define test (lambda (case expr ) 
                 (begin
                 (if (eqv? expr #t) 
                     (write (cons case "complete" ))
                     (write (cons case "failed" ))
                 ) (newline)
                 )))
;execute testCase:
(test "fib is fibonacci function:" (equal? (fib 8) 21) )

Bresenham's circle algorithm

Instruction

Bresenham's circle algorithm, also called midpoint circle algorithm, is an algorithm used to determine the points needed for drawing a circle.

The reason of using this algorithm is, when drawing circle, we need to calculate the floating point numbers of each coordinate points, which may be slow when processing multiple objects.

This method provides a simplified way to draw the circle by estimating the midpoint of arcs.

The Algorithm

for each point p(x,y)
we can take the next approximate points N(x,y+1) , NW(x-1,y+1) by estimating midpoint M(x-1/2,y+1)
that is: if the point M is in the circle, the point N is closer to the circle, therefore the N should be chooses for next point, otherwise, the NW point should be chooses.

so, we can calculate the distance from M to circumference by the formula:
Rd = r - Rm
if Rd < 0, M is outside the circle, if Rd > 0, M is inside the circle.
from x^2 + y^2 = r^2,
we can know that
Xm^2 + Ym^2 - Rm^2 = Xm^2 + Ym^2 - (r - Rd)^2 = 0
therefore, we can Defined D(M) = Xm^2 + Ym^2 - r^2 = Rd^2 - 2Rd = Rd(Rd -2)
because midpoint will be close to the circle, we can ignore the case when Rd > 2.
So if D(M) > 0,Rd < 0 ,M is outside the circle , NW should be chooses if D(M) < 0,Rd > 0 ,M is inside the circle , N should be chooses

We can get the algorithm as below in javascripts:

var drawCircle = function(ctx , x , y , r){
  var cx = r;
  var cy = 0;
  //init draw 4 points on top,right,left and bottom.    
  init(ctx,x,y,r);
  
  //we only need to calculate 1/8 of the arcs, 
  //that is when angle = 45degree, cx = xy 
  while(cx > cy){
    if(cx*cx + cy*cy -r*r > 0){ // D(M) , if D(M) > 0, NW(x-1,y+1) is chooses
      cx--;
    }
    cy++;

    //draw point and mirrow the points to other 7 places on circle
    mirrowing(ctx,x,y,cx,cy);
  }
}

Optimize

The algorithm is fast now and without any floating points,
but we can optimize more by simplify the calculation of D(M)

for each D(M) = D(x-1/2,y) = x^2 - x +1/4 + y^2 -r^2
the next D(Mnew) = D(x-1/2,y+1) = x^2 - x +1/4 + y^2 +2y +1 - r^2
so we can get the difference of each level of D(M), dy = 2y + 1
let D(Mnew) = D(M) + 2y + 1

also, don't forget when the D(M) > 0 , we should move x too,
so D(Mnew) = D(x-3/2,y+1) = x^2 - 3x +9/4 + y^2 +2y + 1
= D(M) -2x +2 +2y +1
so we can get the difference when x moved: dx = -2x +2

for the optimize above we can get the new algorithm:

var drawCircle = function(ctx, x, y, r){

  // d start from D(M) = r^2 - r^2 + r-1/4 = r- 1/4 , but we can ignore constant because r>1/4
  var d = r; 
  var dx = (-2) * r; //dx = -2x +2 , constant move to loop
  var dy = 0; //dy = 2y + 1 , constant move to loop

  var cx = r; //starting point:
  var cy = 0;
    
  init(ctx,x,y,r);

  while(cx > cy){
    if(d > 0){
      cx--;
      d += (-2)*cx + 2;
    }
    cy++;
    d += 2*cy + 1;

    mirrowing(ctx,x,y,cx,cy);
  }
}

you can watch the demo on javascript canvas here.(firefox and chrome/safari only)

ref:
Wiki
http://www.ecse.rpi.edu/

[Style Guide]Google Javascript Style Guide的結語

BE CONSISTENT.

If you're editing code, take a few minutes to look at the code around you and determine its style. If they use spaces around all their arithmetic operators, you should too. If their comments have little boxes of hash marks around them, make your comments have little boxes of hash marks around them too.

"如果妳在修改程式碼, 花幾分鐘看一下其他程式碼並了解它的風格.
如果他們在算數運算子周圍用了空格,妳也該用,如果他們用橫線圍住註解,妳也該用橫線圍注你的注解"

The point of having style guidelines is to have a common vocabulary of coding so people can concentrate on what you're saying rather than on how you're saying it. We present global style rules here so people know the vocabulary, but local style is also important. If code you add to a file looks drastically different from the existing code around it, it throws readers out of their rhythm when they go to read it. Avoid this.

"這個guideline的重點是提供一份共同的字彙庫,讓每個人能專注在妳說什麼而不是妳怎麼說的.我們提出了一份共通的style規則讓大家能了解程式碼的規則,但是小部分的風格也很重要,如果妳的程式碼和周圍的檔案看起來完全不同,這會讓看的人失去了他們閱讀程式碼的節奏感,要盡量避免"

[Leader]領導者的六項特質

在別人的blog上看到的,轉自哈佛管理世界
留在這裡做個分享:

一：沉穩

（1）不要隨便顯露你的情緒。

（2）不要逢人就訴說你的困難和遭遇。

（3）在徵詢別人的意見之前，自己先思考，但不要先講。

（4）不要一有機會就嘮叨你的不滿。

（5）重要的決定儘量有別人商量，最好隔一天再發佈。

（6）講話不要有任何的慌張，走路也是。

二：細心

（1）對身邊發生的事情，常思考它們的因果關係。

（2）對做不到位的執行問題，要發掘它們的根本癥結。
（3）對習以為常的做事方法，要有改進或優化的建議。

（4）做什麼事情都要養成有條不紊和井然有序的習慣。

（5）經常去找幾個別人看不出來的毛病或弊端。

（6）自己要隨時隨地對有所不足的地方補位。

三：膽識

（1）不要常用缺乏自信的詞句

（2）不要常常反悔，輕易推翻已經決定的事。

（3）在眾人爭執不休時，不要沒有主見。

（4）整體氛圍低落時，你要樂觀、陽光!

（5）做任何事情都要用心，因為有人在看著你。

（6）事情不順的時候，歇口氣，重新尋找突破口，就結束也要乾淨俐落。

四：大度

（1）不要刻意把有可能是夥伴的人變成對手。

（2）對別人的小過失、小錯誤不要斤斤計較。

（3）在金錢上要大方，學習三施（財施、法施、無畏施）

（4）不要有權力的傲慢和知識的偏見。

（5）任何成果和成就都應和別人分享。

（6）必須有人犧牲或奉獻的時候，自己走在前面。

五：誠信

（1）做不到的事情不要說，說了就努力做到。

（2）虛的口號或標語不要常掛嘴上。

（3）針對客戶提出的"不誠信"問題，拿出改善的方法。

（4）停止一切"不道德"的手段。

（5）耍弄小聰明，要不得！

（6）計算一下產品或服務的誠信代價，那就是品牌成本。

六：擔當

（1）檢討任何過失的時候，先從自身或自己人開始反省。

（2）事項結束後，先審查過錯，再列述功勞。

（3）認錯從上級開始，表功從下級啟動。

（4）著手一個計畫，先將權責界定清楚，而且分配得當。

[Timeless way of Building] Pattern and Code Convention

Code Convention 是一連串Pattern的集合

Pattern是從許許多多解決問題的方法中,所找出來的,具有共通性的解法
Pattern是從context中誕生出來的,並由人命名而成為design pattern

Timeless way of building中,提到了好的建築中會有的Pattern
包括窗戶的大小,坐北朝南,房間的位址等等.
這些Pattern並沒有名稱,但是卻讓建築更為美好而偉大
我們稱為Quality Without A Name
而好的code也必須要有這些QWAN

1.一致性 : 大雜院 vs 社區

在code project中,我們時常會看到多種不同的寫法,
難以理解的命名與無法找到檔案的目錄架構
有統一的Convention可以讓檔案結構更容易理解
ex : 美國街道的street vs avenue,street為東西向,avenue為南北向,讓人一看就知道這條路往哪個方向
ex : 傳統的專案架構: doc,bin,obj,src...
ex : Pascal , _Camel等命名方式

2.更好的寫法Pattern與設計的指導原則

ex:使用多個if()來取代巢狀的if-else
ex:制定exception handle的規則
ex:遇到不知道的情況以簡單的方法優先
ex:使用String.isNullOrEmpty()來檢查空字串

3.讓犯錯變得困難
制定checkin policy : 於SourceControl設定code的檢查原則,要違反必須要設定attribute
使用using的關鍵字來管理連線,避免浪費連線資源
(Javascript)使用(function())()這種closure pattern來保持namespace的清潔

[Translation]The SQL high performence series part II

偷懶了兩個月沒有寫文章,目前差不多也適應了新的環境
就以利用空閒時間翻譯的這篇文章作為blog復活的開始吧

原文:The SQL high performence series part II
這篇文章是三篇系列文的第二篇(不過作者於第三篇富監了),裡面對索引以及資料庫是如何決定搜尋計畫有不錯的說明

基本?我們可都是專家!

大部分於這篇文章中提到的內容對許多開發者都是老生常談, 但還是值得複習 – 即使在專家之間對於資料庫的基礎,以及影響系統表現的重大原因,都還是留有誤解以及矛盾的資訊. 我將這篇文章寫的比較像一篇演講,而不是一張」重要清單」,但如果你只想看結論可以直接按End

Data Size

這需要在這裡被強調是因為有越來越多的"data wasters" 錯誤的相信只要他們浪費越多,這個專案就會越像一個"企業解決方案",我跳入這個泥沼,知道有些賺飽口袋的反對者會有"你在說謊"之類的回應,不過如果這樣做能夠多省下一byte tree的資料,我願意付出這個代價.

最小化你的資料,當你不需要GUID的時候,不要用GUID(例如:你不需要做跨全域的複製以及不重複的ID時),不要用bigint當你只需要int或small int, 不要用small int 當一個 tiny int就可以滿足需求,用varchar當你不需要nvarchar.使用最小並適合的資料型態,不要到了要刪減過大的資料時,才使用不明確的packing技巧或是原生資料型別.

當然你應該計畫現實上有可能的成長, 而且我並不是在鼓吹你需要用一個tinyint來儲存你的CustomerID欄位,但是在設計你的應用程式時要有一個合理的標準 – 你真的會有超過兩億的使用者嗎?你的應用程式會有32767種語言嗎? 有可能我們會面對一個需有兩億個月份的日曆系統嗎?

去估計在資料庫中使用這些大類別儲存較小的資料是不是可接受的妥協,這會增加效能,並且允許你能簡單的升級你的資料型別,當那些不太可能發生的需求成真的時候。

很明顯的有需多案子確實需要大資料型別的保障,但太多DBA用GUID或BigInt儲存資料以防萬一,來逃避效率上的責任(建立GUID也要花費資源,並且會增加儲存以及I/O的花費,當GUID被用來記錄網路卡的MAC位置時,GUID是循序性的,但現在Windows上的GUID只是單純的亂數, 作為叢集主鍵將會導致無止境的資料重新排列)

這有什麼重要的?在實際的企業應用程式中,有好幾百萬的資料列在資料庫中,而又需要從像冰河一樣緩慢的儲存系統中讀取與寫入資料 – 在這些有限的資源中, 使用1MB來傳輸20000筆記錄不是比5000筆記錄來的好嗎? 10%的資料庫可以儲存到快取記憶體是不是比5%好呢?當然.

不要被那些小規模,所有資料都在記憶體而Query中的計算大於I/O費用的Benchmark所騙. 認為使用大的DataType隻影響了10%左右的表現 – 當你的資料庫進入真正的企業層級時, 尺寸真的很重要(size really does matter). 使用GUID當叢集式索引的主鍵不僅讓你的資料列更大, 也讓非叢集索引更大(並更慢), 當你SAN中最弱的一環正以100%在跑,你會後悔你浪費的每一個byte

索引

許多SQL Server表現上的問題都是源自於沒有或不正確的索引, 或是沒有用到的後補索引. 這常發生在前端專家不情願的接下資料庫的工作時,甚至是發生在由資料庫專家精心打造的資料庫中。
瞭解索引, 並專注於使用它們 , 是高效能資料庫最重要的一點. 不只是建立正確的索引, 還有如何正確的調整現有的索引.

非叢集索引

在參考書中的索引和資料庫中的索引有同樣的意義(並有很多相同的特徵)，參考索引你可以快速的找到你想要的資訊，與掃過書裡的每一頁來找尋內容不同，可以直接找到你想要的頁面。在SQL Server的世界中這種索引稱為非叢集索引(non-clustered indexes或是Secondary indexes) - 他們是一個依特定目的而排序資料的子集合，包含了一個指向實體資料列的指標。

在Northwind 資料庫中Order Table的ShipPostalCode是一個非叢集索引的範例。這個索引由ShipPostalCode排序，並可能在下列的Query中使用。

SELECT * FROM Orders WHERE ShipPostalCode = '05022'

如果你看一下執行計畫(當你執行Query時按Ctrl + K 或點選"顯示執行計畫"，執行計畫會出現在下方的結果分頁中)，你可以看到索引查找(index seek)的發生，並執行一個Bookmark lookup來找到原始的資料列。如果我們在執行以上的Query前先執行SET STATISTICS IO ON，我們會得到此Query的IO使用量統計報告，會像以下這樣:

Table 'Orders'. Scan count 1, logical reads 4, physical reads 0, read-ahead reads 0.

比較不會使用到索引的以下Query:

SELECT * FROM Orders WITH(INDEX(0)) WHERE ShipPostalCode = '05022'

在這個案例中執行計畫顯示了完整的資料表搜尋(full table scan)，IO統計報告如下:

Table 'Orders'. Scan count 1, logical reads 21, physical reads 0, read-ahead reads 0.

在沒有索引的情況下明顯的使用了更多I/O,如果資料量增加到企業級的數量的話,這情況會變得更糟.讓這個事情更糟的是,每一次的資料表搜尋(table scan)會產生Page/row lock, block掉整個資料表的讀取,讓接下來的交易必須等待資料被unlock之後才能確定需要的資料在不在那些被鎖定的資料之中.可是索引查找(index seek)能知道被鎖定的資料並不是他要搜尋的資料,所以不會被影響.試試看同時在兩個分開的查詢視窗中跑兩句query(如果你不能夠在30秒內很快的切換查詢視窗,可以增加WAITFOR的延遲)

BEGIN TRAN

UPDATE Orders SET OrderDate = '1997-08-27'
WHERE OrderID = 10647

WAITFOR DELAY '00:00:30'

ROLLBACK TRAN

第一句使用索引的Query, 會忽略row lock資料列馬上回傳結果,因為被鎖定的資料列並不是這個query所要尋找的. 但第二句沒有使用索引query(可能因為沒有建立索引,或索引不是最佳搜尋選擇),會被block到另一個連線的鎖定解除. 不使用索引不只是非常的沒有效率,當資料庫增加規模時,也可能讓blocking問題顯著的惡化(或在嘗試增加規模時)

叢集式索引

回到我們的譬喻,許多書更進一步的將他們的內容排序,讓資料本身就是排序好的索引,一本料理書可能是按照料理名稱排序,或是一本電話簿是按照城市,姓,名來排列.所以如果你想在這些排好序的資料中搜尋,是非常的有效率的 - 在電話簿的例子中,你可以很快的找到你想找的城市以及姓名,然後在一小筆的資料中掃瞄你想找的人.在SQL server中這種排序叫做叢集式索引.(資料本身的排序),並且有很明顯的原因在一本書或一個資料表中只能有一個叢集式索引.它主要的優點就是所有符合索引的資料都可以迅速的提供,不用其他的參照.

在Northwind資料庫中使用下列的Query:

SELECT * FROM Orders JOIN [Order Details] ON Orders.OrderID = [Order Details].OrderID WHERE Orders.ShipPostalCode = '05022'

如果你檢視執行計畫,Order Details的資料會由一個有效率的索引查找(index seek)來取得.
不過叢集式索引並不是萬靈丹. 例如說,想像你是一個努力工作的打字員,正在維護一本電話簿的排版,然後你小心的將每一筆資料放在正確的頁面上.每次一筆新的資料進來,它並不會正好就符合排序資料的尾端,或是有時候改變已經存在的資訊會同時改變它的排序(例如說Smith先生改名成Jones),你需要重新排列一些頁面已取得空間.這些資料碎片的問題在非叢集與叢集索引都會發生, 但叢集式索引由於包含所有資料,所以這個問題特別嚴重.

當然你可以在每一頁預留一些空間放置這些資料來讓一些小的更動容易些,也就是在SQL Server中fillfactor(填滿因數)的目的(一個較低的fillfactor會在分頁留下更多的空間但會減低真正的資料密度 - 插入資料的速度增加,但讀取資料的表現則減少. 一個較高的fillfactor增加資料密度並提升讀取速度,但增加插入操作時會造成分頁的可能性.注意fillfactor只在索引建立時與重組/重建索引時有用),不規則的插入資料或經常改變叢集式索引欄位可能是會一個嚴重的效能問題.因為這個問題很多開發者使用單一的遞增ID 欄位來作為叢集式索引.歷史上來說有個原因是因為有複數且相同的欄位資料作為索引插入時,全部會被放到相同的索引底下,結果會在這個熱點導致資料的爭奪.SQL Server有邏輯可以解決這些ID欄位並有效的消除這些熱點的問題.

另一個叢集式索引的問題是他們的長度(他們包含了整個資料列的資料). 當你在找某一個特定的資料欄位時,其他資料有可能並不相關,或是你真的計畫在查找後使用所有的資料,但如果你查的是一個範圍內的資料(例如說在Oakville city姓Forbes的人的所有名子)資料庫引擎會使用範圍掃瞄(range scan)讀取整行的資料來找尋符合的資料,再抽出需要的資料. 在我們的電話簿例子中這小小的額外資料並不算什麼,但在真實世界的大型的資料表上可能會導致效能上的衝擊.
考慮如果我們有第二個用city,姓,名來排序的索引,資料庫引擎可以很有效的搜尋這些較小的索引內容.

覆蓋索引(Covering Indexes)

這個帶來一個重點 - 有些索引本身就包含足夠的資訊使你不需要去讀取內容,你的搜尋只要讀取索引就能滿足需求了.考慮一本旅遊書的索引,將有名的景點以國家,城市與名稱排序,然後將你導向更多資訊的頁面,如果你只是想知道義大利的Accademia dell' Arte del Disegno在什麼城市,快速的搜尋索引就可以知道他在Florence. 在這個例子中索引為"覆蓋索引(covering index)",包含了可以覆蓋需求的所有資訊,讓我們不需要去查找資料列來找尋需要的資訊,這常常是最有效的搜尋機制.

變化之前範例中的query:

SELECT OrderID FROM Orders WHERE ShipPostalCode = '05022'

這會很有效的使用ShipPostalCode索引來找尋特定的紀錄,並且這個索引完全可以滿足query本身,所以可以避免消耗資源的bookmark lookup,IO也可以被最小化.

Table 'Orders'. Scan count 1, logical reads 2, physical reads 0, read-ahead reads 0.

有人應該已經觀察到OrderID實際上並不在索引ShipPostalCode裡,或至少不應該在.但有趣的是所有已排序的資料中的叢集式索引欄位,會自動被加到其他索引的欄位中. 當你需要一個叢集式索引欄位的資料時,有可能非叢集索引的資料會立即滿足你,但也必須衡量它會讓每個其他的索引更大,導致更低的資料密度.

小型的覆蓋式索引是取得資料最有效率的方法,也是一個讓你確認你只取出你實際需要資料的理由.範圍掃瞄(Range Scan)在很多情形下也是很有效率的,並且通常在需要的資料在索引中是連續的的時候,像是你用BETWEEN搜尋兩個日期之間的資料時.雖然因為之前提過的bookmark lookup的花費, 範圍掃瞄(Range Scan)只在索引能完全覆蓋或是使用叢集式索引來搜尋的時候才能看到.

即使在完整的搜尋為必要的時候,索引仍然可能完全覆蓋,考慮加下來加入Customer table的索引:

CREATE INDEX CountryCity ON Customers(Country,City,Address)

這個索引當然包括了Country,City,Address,但如之前提過的它也包含了CustomerID因為它是叢集式索引.單然如果我們搜尋Country,或是Country與City,或是Country,City和Address一個很有效的搜尋或是範圍掃瞄(Range Scan)可能會用來找出符合的資料.

SELECT CustomerID FROM Customers WHERE Country = 'Canada'

如果我們只搜尋address呢?在這個情況下索引不能被用到因為他是先以country再以city與address排序的,所以一個特定的地址有可能出現在索引的任何位址

SELECT CustomerID FROM Customers WHERE Address = '43 rue St. Laurent'

你可能會很驚訝看到它仍然使用了索引, 儘管這次用的是很沒效率的掃瞄(scan)而不是查找(seek).索引因為覆蓋了query所有的敘句和傳回欄位而被使用,且因為索引只是資料的子集合,和資料表掃瞄(scan data table)比起來它使用較少的I/O來掃瞄整個索引

統計資料與bookmark

在之前的例子上我們跑了一個需要bookmark lookup的query(因為他沒有覆蓋索引).query如下:

SELECT * FROM Orders WHERE ShipPostalCode = '05022'

如果你觀看這個query的執行計畫可以看到它在索引中找尋"lookups",並且在叢集式索引上進行bookmark lookup,找尋真正資料列的資料.在這個案例中只有單一的傳回資料列,但即使如此,bookmark lookup的費用還是佔了50%的query運算費用.

bookmark lookup 的花費,是當一個item在索引中被找到,但索引並沒有包含所有需要的資料時所產生的,這也是為什麼很多人驚訝的發現SQL Server忽略了他們認為是完美的索引而使用資料庫掃瞄 (為什麼他們不用我的索引啊啊啊啊!!!). 考慮以下的query

SELECT * FROM Orders WHERE ShipPostalCode = '24100'

從執行計畫你可以看到實際上進行了一個叢集式索引掃瞄 (是用叢集式索引進行的一個資料表掃瞄)來取代使用我們的索引,而且subtree的花費是0.0530
這也許看起來有些複雜,因為我們應該已經有了一個完美的索引,但讓我們重新執行一次相同的query,這次使用一個query hint來強迫他使用我們的索引

SELECT * FROM Orders WITH(INDEX(ShipPostalCode)) WHERE ShipPostalCode = '24100'

你可以從執行計畫上看到它使用了我們的索引,但這次bookmark lookups佔了所有執行時間的80%.我們的subtree花費變成了0.0564 - 比使用資料庫掃瞄還多!

在這個案例中只使用了總共830筆中的10筆記錄(1.2%),而它仍然選擇執行資料庫掃瞄而不是我們的索引.很多開發者都被這個情況迷惑,不知道為什麼SQL Server不使用他們美麗的索引,但資料庫引擎是因為非常實際的理由這樣做的 - 因為這樣比進行非直接從bookmarks的找尋每一小塊資料來的有效率.

當然我們可以儘可能的讓覆蓋索引包含所有必要的資訊來避免bookmark lookups,假設它包含了我們需要從資料表中取得的所有資訊:

SELECT OrderID FROM Orders WHERE ShipPostalCode = '24100'

現在它會使用我們的索引,而且超級有效率. 它的subtree花費只有0.0064.在大型的資料庫這些迴避bookmark lookup與資料表掃瞄的差異可能會非常大.

所以資料庫引擎是如何猜到有多少資料列會符合標準,來決定使用哪個方法最有效率?(不論是索引,bookmark lookup或資料表掃瞄) 這就是分散式統計資料(distribution statistics)派上用場的時候了. 統計資料(Statistics)是一個資料集合,資料庫引擎用來推測哪個計畫在這個查詢上最有效率.你可以用DBCC SHOW_STATISTICS 的指令來看到某些索引的統計資料,在以下的例子中為ShipPostalCode:

DBCC SHOW_STATISTICS('Orders','ShipPostalCode')

因為資料表中的ShipPostalCode是有範圍並分散的號碼,所以在這個例子中統計資料是完全正確的,在一個更實際的資料庫中,有上百萬行的資料列, 統計資料開始更接近估計(錯誤邊際值會逐漸增加),這些資料庫引擎的估計在某些極端的情形下可能會導致完全錯誤的假設,像是預測會產生幾千行資料但實際只用了幾行.

統計資料也可能在複數索引時出錯.在這個案例中可選擇的第一個欄位被使用到,所以之前我們建立的索引(Country,City,Address)因為Country的低辨識性(有很多相同的Country欄位),這個索引經常會被忽略,即使City與Address的結合是非常unique的. 所以普遍來說都建議使用可辨識性最高的欄位作為索引的第一個欄位,在這個案例的索引來說是Address,City,Country. 這是可以討論的因為它讓索引更具有單一的用途-他不再具有增進效率的用途,提供給比較模糊的搜尋像只有Country,或是Country/City. 但這需要依照案例來判斷,而且在一個完整正規化的資料表中,這並不是個問題.

並且需要注意的是,你的統計資料儘可能的接近正確值是很重要的. SQL Server包含了自動更新統計資料,依照預設值,在覺得資料已經過期時會自動嘗試資料取樣與更新統計資料. 但儘管如此,最好的方法還是定期的安排完整的統計資料更新(最少每週),最好使用WITH FULLSCAN 的選項讓結果儘可能的正確. 標準的database維護計畫包含了更新統計資料的步驟,允許你選擇取樣的資料範圍.

但即使依照以上的步驟,你的統計資料仍然有過期的可能. 在這種情況下可能是索引提示需要被加入query,來要求query重新評量. 不過很明顯的這應該是最後才考慮的手段.

實際使用索引

所以你已經建立了美麗的索引,然後你已經確定了你的query只從每個資料表中提取了必需的資料,在可能的地方使用覆蓋索引來避免bookmark lookup的花費. 你使用執行計畫來評量效能,並發現......資料庫引擎完全忽略了你的索引. 以下幾點是可能的原因.

考慮以下的query

SELECT OrderID FROM Orders WHERE LEFT(ShipPostalCode,4) = '0502'

很簡單的query, 而且看起來這像是一個很有效率的覆蓋索引查找(covered index seek). 但執行後會發現這其實是一個沒有效率的掃瞄.考慮下列query:

SELECT OrderID FROM Orders WHERE ShipPostalCode LIKE '0502%'

在這個案例中query以高效率的索引查找(index seek)執行. 我已經有過這種微小的差距讓企業報表產生的時間從幾小時降到幾秒鐘的案例.

原因是因為前一個敘句的索引欄位被隱藏在函式之中. 資料庫引擎不能夠預測函式的結果是什麼,所以他會強迫去計算每一列的來看產生的結果. LIKE是第一級的比較函式,資料庫知道他的行為,所以可以對他做最佳化. 有無數的案例是由於人們沒有必要的將索引欄位放到了函式中,造成了大量與無法預期的效率低落.

最常見的例子是使用DATAADD/DATEDIFF來取得一定時間內的資料列 - 而不用預先計算好的數值(例如: GateDate() - 3 years) 並與資料列進行直接的比對, 開發者強迫整個資料庫掃瞄在每一個資料列上進行無謂的資料比對. 考慮一個從新聞資料表報告12小時內所有新出現新聞的query.

DECLARE @CurrentTime 

SET @CurrentTime = GetDate() 

SELECT * FROM NewsStories WHERE DATEDIFF(hh,NewsDate,@CurrentTime)<12

我保證這會非常的沒有效率, 但這也是非常普遍的例子. 使用下列變數,資料庫引擎可以更有效的最佳化:

DECLARE @StartTime   
SET @StartTime = DATEADD(hh,-12,GetDate())   
SELECT * FROM NewsStories WHERE NewsDate > @StartTime

資料庫速查表:

索引欄位

在文章開頭我建議你們最小化使用的資料空間(增加實際資料密度). 這個目標並不是要將資料庫移植到一張磁片上,而是最小化查詢計需的磁碟I/O,因為磁碟I/O是企業系統中最脆弱的一環. 當然增加而外的索引,是一個設計上的選擇.它雖然會增加資料庫的大小但卻能減低實際上需要存取的I/O,所以通常都是很值得的交換.

另一個有用的技巧是,你可以以磁碟空間交換資料庫的效能,有很多的變化,但我只提一個最常使用的情形 - 依照年份提供每個月份的報表. 在這個情形下Orders 資料表可能被壓縮成用以下的Query來查詢:

SELECT YEAR(OrderDate) AS [Year], MONTH(OrderDate) AS [Month], COUNT(*) AS [Monthly Orders] 
FROM Orders 
WHERE YEAR(OrderDate)=1997 
GROUP BY YEAR(OrderDate), MONTH(OrderDate)

取代使用串接式的命令來取得年分與月份的資料,考慮把它們加為資料欄位:

ALTER TABLE dbo.Orders ADD 
OrderDateYear  AS CONVERT(smallint,YEAR(OrderDate)), 
OrderDateMonth  AS CONVERT(tinyint,MONTH(OrderDate))

現在我們可以把query改成

SELECT OrderDateYear AS [Year], OrderDateMonth AS [Month], COUNT(*) AS [Monthly Orders] 
FROM Orders 
WHERE OrderDateYear=1997 
GROUP BY OrderDateYear, OrderDateMonth

本身我們並沒有用任何方法來增加查詢的效率(事實上查詢效率在變更之後還降低了),雖然我們讓資料庫更複雜化,但我們也建立了一些索引欄位作為建立索引的基礎.

CREATE NONCLUSTERED INDEX IX_OrderDateDecomposed ON dbo.Orders
(
OrderDateYear,
OrderDateMonth
) ON [PRIMARY]
GO

現在參考這些索引欄位的查詢戲劇性的便的非常有效率,更好的是,這些增加的索引欄位並沒有減低實際的資料密度,因為他們只出現在索引中. 注意:確認不需要這些索引欄位的查詢不會外在的或隱含的使用浪費的* column selector, 因為它會不必要為每一個資料列評估這些欄位.

索引視圖

[這個功能只在Enterprise和Developer edition of SQL Server才有]

一個索引視圖,有時候代表了一個具體的視圖,是一個排列後的索引欄位,把儲存計算結果帶到了下一個階段.在一個之前的例子中我們將索引欄位組合在一起來增加資料組合的邏輯效能.我們可以把它做個改變來建立一個視圖(View)

CREATE VIEW dbo.OrdersByMonth
WITH SCHEMABINDING
AS
SELECT OrderDateYear AS [Year], OrderDateMonth AS [Month], COUNT_BIG(*) AS [Monthly Orders]
FROM dbo.Orders
GROUP BY OrderDateYear, OrderDateMonth

這個視圖(View)只是一個查詢的範本,並且只對重用程式碼有用(雖然也是個很有價值的目標).我們可以更進一步的實體化這個視圖(View),儲存結果並更改它,使更改內容能反映到實際的資料.以下的指令可建立一個索引視圖:

CREATE UNIQUE CLUSTERED INDEX IX_OrdersByMonth ON OrdersByMonth(Year,Month)

現在以下的查詢可以滿足索引視圖了,使用預先計算的值來取代每次查詢的重新計算.

SELECT [Year],[Month],[Monthly Orders] FROM OrdersByMonth WHERE Year=1997

最後我們每個月的訂單查詢使用的資源比原本的下降了90%,這還只是在一個很小的sample資料庫的結果.在真實的資料庫中結果可能差距會非常大.

索引視圖的確有缺點,像是會在資料改變的時候執行自動維護,但這可以是一個非常有用的工具,當你的工具箱中有這項工具並且你使用的平台支援它的時候.

結論 :

儘可能保持資料列很小, 增加資料密度
所有的Table都需要叢集式索引,除了一些例外. 通常單一欄位的主鍵就是索引
小的叢集式索引讓非叢集式索引也很小,增加資料密度
叢集式索引可幫助建立其他覆蓋索引
使用覆蓋索引非常的有效率
避免把標準欄位隱藏在函式中 – 他們不能用到索引
當適合的時候使用有索引的欄位
如果你已經花錢買了企業板的資料庫,認真考慮使用索引視圖(indexed view)
索引索引索引!只有很有限的案例中,新增與插入時維持索引的費用會超過它的好處
瞭解執行計畫,定期的去評估它

[.Note]jquery動態調整欄寬&限制checkbox選取

幾個專案上遇到的小需求,
皆用JQuery來解決,在這裡記錄一下解法

1.動態調整欄寬 :

有一個table,他的第一個欄位長度是固定的,後面的欄位要有相等的寬度,
但是後面的欄位數目是動態產生,欄位的名稱長度也不固定,如果欄位過長則必須要換行

一開始是用%數的方法來指定欄寬長度,但是這個方法並不能夠有效的在所有瀏覽器上運行
後來使用jquery來動態調整欄寬 : 先等table產生,再計算出後面欄位的平均寬度,並將儲存格寬度調整為平均寬度:

$('.sort').each(function() {
    var sum = 0;
    //取得所有要調整的欄寬
    var cells = $(this).find('.sort-item');
    //統計總寬度
    cells.each(function() {
        sum += $(this).width();
    });
    //設定平均欄寬 
    var cellWidth = sum / cells.length;
    cells.each(function() {
        $(this).width(cellWidth);
    });
});

2.限制checkbox選取

有一個多選的CheckboxList,需要限制最大的選取數量
如果超過則不能選擇:

//傳入CheckboxList Container的ID
bindMultiSelectLimit = function(id,limit) {
    $('#' + id + ' input:checkbox').click(function() {
        //使用Jquery selector直接找出所有已選之Checkbox
        if ($('#' + id + ' input:checked').length > limit) {
            return false;
        }
    });
};

3.限制排序問題重覆選取

有一個問卷題型叫排序題,結構上是由許多RadioButtonGroup所構成,
讓使用者能排列喜好選項的順序 : (ex, 有三個選項, 可以選擇 3,2,1 或 1,2,3的排序 )
但使用者希望能夠有防呆機制,讓已經於其他選項選過的排序不能再選第二次
(ex:三個選項的排序可為 1,2,3. 3,2,1 . 但不可為 2,2,1 )
後來選擇了一個比較簡單的作法:當排序被選取時,清空其他有同樣排序值的RadioButton

//傳入ContainerID
bindSortRules = function(id) {
    $('#' + id + ' input').click(function() {
    //設定每個排序RadioButton有同樣的value,將其他有相同value的值都設為false
    $('#' + id + ' input[value=' + $(this).val() + ']').attr('checked', false);
        //設定目前選擇的值
        $(this).attr('checked', true);
    });
}

以上的需求如果使用Javascript不知道要做多久...
所以說JQuery真的是能夠有效增加前端生產力的一個好框架啊,
只要有用到Web的開發者真的都需要接觸一下

[soapUI]如何測試你的WebService

當我們開發WebService時,有一個問題就是我們要如何去測試WebService是否能成功的被呼叫，並且如何去建立測試案例來確認讓服務可以工作。

soapUI提供了完整的WebService相關測試功能以及介面,讓開發與測試WebService變得更容易
包括了呼叫WebService，建立測試案例, LoadTest ,建立Mock Service讓程式呼叫等等

在使用SOAPUI之前，須先對WebService的一些基本架構有些了解 :

WebService透過了WSDL (Web Service Definition Language)定義服務提供的介面，並經由架構在Http上的SOAP(Simple Object Access Protocal)，來傳輸資料，所以在使用WebService之前，須先從url獲得服務定義的wsdl檔，再以SOAP呼叫服務，獲得回傳結果。