Posted by Seon
on July 10, 2008
This is probably old news, but I still get excited thinking about all the interesting solutions that are possible now using Mule and Gigaspaces for SOA problems requiring low latency and highly scalable architectures.
Gigaspaces released 6.5 with API integration with Mule 2.0 … this is just plain awesome. You can use Gigaspaces as the transport (e.g. in place of JMS) and quickly get a SBA up and running utilizing the same concepts I used at RHG when we were servicing B2B problems. You also get the advantage of the clustering ability and fault tolerance that comes with Gigaspaces – which is just pure sex – not to mention all the other great features that come with this advanced Javaspaces implementation (i.e. management tools, monitoring tools, data partitioning, performance features like batching).
Gigaspaces would have made a fantastic backend transport and shared data grid in place of ActiveMQ JMS, but at the time it wasn’t as fully integrated with Mule. We would have been able to solve alot of the high availability and clustering concerns regarding single points of failure with the messaging database and JMS brokers. I never felt comfortable with ActiveMQ’s clustering and fail-over features. Configuration seemed a bit fussy and it doesn’t seem to be as easy to scale out as a Gigaspace cluster.
Posted by Seon
on July 09, 2008
Google’s awesome, they release some nifty software to the open source community. Here is the link to Google’s Protocol Buffer and more details are available on Google’s Open Source Blog
From what I can tell, it’s a solid format that can replace XML in most scenarios. It’s not as easily human-readable as XML, but then again from experience not many humans are actively reading the XML anyways. The proto format reminds me of a json structure, minus the curly brackets. Also, from skimming the proto java documentation there isn’t support for primitive arrays (at least for java). You should be using List containers instead along with the proto “repeated” scalar.
They include code for data binding source .proto files to C++, Java and Python. This makes it easier to adopt in your next project. I think this is a good data format for internal use, but you’ll probably still have to deal with XML/SOAP and other RPC formats (JSON, etc) on your external interfaces. This would require transforming the internal data format (.proto) into the external format. There is also stub generating code for producing server interfaces that you can drop into your native rpc server implementations. This is pretty cool also.
Hmmm, I see some useful Mule and Camel transformers that could be made to take advantage of this new format. And possibly service adaptors for Mule and Camel too.
Posted by Seon
on June 19, 2008
I’ve encountered a typo in the JMS package from Oracle while I was debugging a problem with multicast routing under Camel and Oracle AQ. It seems that Oracle may have slipped up and misnamed the “JMSXRcvTimeStamp” property as “JMSXRecvTimeStamp”. I don’t have access to the source code for the ORacle’s implementation of javax.jms.Message (AQjmsMessage), however I poked through the class file with a hexeditor and found instances of the typo.
According to the Java JMS 1.1 specification, this is a property that is set by the jms provider when a message is dequeued. The spelling of the property is listed as “JMSXRcvTimeStamp” on page 39 of the spec pdf. Oracle Streams Advanced Queuing User’s Guide and Reference
10g Release 2 document referencfes the correctly spelled property in section 11.2.2 however elsewhere in the same document it is incorrectly spelled. I’m pretty sure this is a bug in Oracle’s implementation of the Message interface. The document for Release 11g shows the same errors.
Additionally, I get the feeling that either Camel, Oracle or Spring is incorrectly setting null values for several standard JMS headers and properties when they are not assigned or not present. If null values are allowable under the JMS spec, then Oracle’s AQjmsMessage.setJMSReplyTo(Destination) should accept null parameters without exceptions also. Right now passing a null Destination value to method setJMSReplyTo(Destination) results in this exception:
|
oracle.jms.AQjmsException: JMS-147: Invalid ReplyTo destination type, or use of reserved `JMSReplyTo agent name, or serialization error with AQjmsDestination org.springframework.jms.UncategorizedJms
|
Here is the post I created in the Camel users forum relating to this issue.
Here is the JIRA issue I created for camel-jms: https://issues.apache.org/activemq/browse/CAMEL-618
Posted by Seon
on March 15, 2008
The Cult of the Dead Cow (cDc) recently released Goolag which in simplest terms is a mass exploit scanner. Their Goolag specifications page has more information and makes for a pretty entertaining read. Though they are humorous, this tool is pretty serious stuff for identifying exploits on a remote host. I could easily find a use for this in my workplace; this could be one of many tools used to audit a website before it is placed into production. It’s better to scan and catch the security holes on your own, before an unknown attacker does it for you.
On a completely unrelated note, I had run across a means of using Google to locate and display live images from video cameras all around the world. This was several months ago and I had said to myself and friends that this would make for a great screensaver – a mosaic of live images from random web cameras from around the world. Well, looks like someone went and implemented it. Check out surveillancesaver which has versions for both Windows and OSX.
Posted by Seon
on December 06, 2007
Turns out the MPAA is trying to install backdoors on students’ computers that monitor and relay information back to the MPAA. Fortunately, people have taken notice and some have even issued a DMCA takedown notice against them. There is alot of irony in this situation and it really highlights the MPAA’s greed and willful ignorance of copyright laws which do not benefit their organization directly.
Posted by Seon
on September 18, 2007
Dan Diephouse (formerly of XFire, now with MuleSource) gave a presentation recently at JavaZone Norway and the PDF and MP3 are now available here.
I haven’t listened to the MP3 but the PDF had some good tidbits and gave some insight into Dan’s approach regarding RESTful services in an SOA environment. He doesn’t go into enough detail for my tastes (I like to see as much nitty gritty detail as possible) but it’s more than just a rehash of management-techno-speak so I appreciated it. There are some sections which are pretty basic (like explanation of SSL). There is also some good information about transactions and why we should approach them differently knowing some technical constraints in the HTTP protocol and stateless web communications. He briefly touches upon security issues also. All in all I wish he went into more detail, but it’s all good; another contribution to the RESTful services discussion and it’s pros/cons.
Update
Dan’s website mentioned in the PDF doesn’t seem to work (http://www.netzooidicom.com/blog).
Posted by Seon
on September 03, 2007
I’ve replaced tcpmon with ngrep as my tool of preference for debugging service interaction while working with Mule. Tcpmon required modifications to service configurations in order pass data through the tcpmon proxies for monitoring. However ngrep is transparent and doesn’t require these configuration changes. It has a simple and fast text interface. No complicated gui to navigate. The output can be piped to grep or saved to file. Run on *nix and Win32. Sweet!
On OSX (version 10.4.10) I decided to compile and install the latest ngrep using the following steps…
1
2
3
4
5
6
7
|
wget http://prdownloads.sourceforge.net/ngrep/ngrep-1.45.tar.bz2?download
tar -jxvf ngrep-1.45.tar.bz2
cd ngrep-1.45
./configure --prefix=/usr/local
make
sudo make install
|
I use ifconfig to determine the interface I want to monitor with ngrep…
1
2
3
4
5
6
7
8
9
|
ifconfig
en1: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
inet6 fe80::216:cbff:feb7:5de%en1 prefixlen 64 scopeid 0x5
inet 172.16.1.3 netmask 0xffffff00 broadcast 172.16.1.255
ether 00:16:cb:b7:05:de
media: autoselect status: active
supported media: autoselect
|
Then startup ngrep on eth1 monitoring port 80. The -Wbyline detects linebreaks in the packet data…
1
2
3
4
5
|
sudo ngrep -Wbyline -d en1 port 80
Password:
interface: en1 (172.16.1.0/255.255.255.0)
filter: (ip) and ( port 80 )
|
Example ngrep output…
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
|
####
T 172.16.1.3:57867 -> a.b.c.d:80 [AP]
GET /usage.html HTTP/1.1.
Host: ngrep.sourceforge.net.
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6.
Accept: HTTP Accept=text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5.
Accept-Language: en-us,en;q=0.5.
Accept-Encoding: gzip,deflate.
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7.
Keep-Alive: 300.
Connection: keep-alive.
Referer: http://ngrep.sourceforge.net/.
Cookie: __utmz=191645736.1188003977.1.1.utmccn=(referral)|utmcsr=jibx.sourceforge.net|utmcct=/mail-lists.html|utmcmd=referral; __utma=191645736.2030039243.1188003977.1188003977.1188003977.1.
.
##
T a.b.c.d:80 -> 172.16.1.3:57867 [A]
HTTP/1.0 200 OK.
Date: Mon, 03 Sep 2007 21:43:32 GMT.
Server: Apache/1.3.33 (Unix) PHP/4.3.10.
Last-Modified: Thu, 24 Feb 2005 04:41:08 GMT.
ETag: "219bb8-4827-421d5ae4".
Accept-Ranges: bytes.
Content-Length: 18471.
Content-Type: text/html.
X-Cache: MISS from sims.sofast.net.
Connection: keep-alive.
.
<?xml version="1.0" encoding="UTF-8"?>
<!doctype html public "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head><title>ngrep - network grep</title>
</head>
<style>
body {
text-align: center;
padding: 20px;
margin: 0;
background: #888;
color: #000;
font: 10px/15px verdana, geneva, helvetica, arial, sans-serif;
}
h2 {
text-align: center;
margin: auto;
width: 600px;
}
...snip...
|
Posted by Seon
on September 03, 2007
A few code snippets using CodeRay filters for syntax highlighting.
lang=”ruby”
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
require 'coderay'
class CodeMacro < FilteredColumn::Macros::Base
def self.filter(attributes, inner_text = '', text = '')
lang = attributes.delete(:lang)
attributes[:line_numbers] = :table unless attributes.has_key?(:line_numbers)
attributes.each do |key, value|
attributes[key] = value == 'nil' ? nil : value.to_sym rescue nil
end
begin
CodeRay.scan(inner_text, lang.to_sym).html(attributes)
rescue ArgumentError
CodeRay.scan(inner_text, lang.to_sym).html(:line_numbers => :table)
rescue
unless lang.blank?
RAILS_DEFAULT_LOGGER.warn "CodeRay Error: #{$!.message}"
RAILS_DEFAULT_LOGGER.debug $!.backtrace.join("n")
end
"<pre><code>#{inner_text}</code></pre>"
end
end
end
|
lang=”javascript”
1
2
3
4
5
6
7
|
fixIEOverlapping: function() {
Position.clone(this.update, this.iefix, {setTop:(!this.update.style.height)});
this.iefix.style.zIndex = 1;
this.update.style.zIndex = 2;
Element.show(this.iefix);
},
|
Posted by Seon
on September 01, 2007
My friend and coworker Gabe (of Slickr and Dclicio.us fame) has recently released a great ruby gem for generating rake tasks and templates for developing Adobe AIR apps. Check it out!
Airrake, Rake tasks and generators for Adobe AIR apps.
Posted by Seon
on August 28, 2007
I’ve hopped on board the Nginx + Mongrel train for my website (mephisto, rails, mysql, mongrel cluster, nginx – deployed with vlad capistrano v2). After some brief wrangling with apache conf files, I took a break and looked over Nginx. Turns out it’s pretty simple to get up and running. The best thing about nginx aside from its small memory foot print (compared to Apache2) is the ability to hot-swap new binaries using unix kill signals. The nginx configuration can also be reloaded on-the-fly. Nginx is a sweet little daemon.
Here are some of the steps I followed to get it up and running on Debian Etch. These instructions assume you know/have already installed the appropriate ruby gems.
Compiling nginx/0.5.31 (updated 09/05/2007)
1
2
3
4
5
6
7
8
9
|
apt-get install libpcre3
apt-get install libpcre3-dev
wget http://sysoev.ru/nginx/nginx-0.5.31.tar.gz
tar xzvf nginx-0.5.31.tar.gz
cd nginx-0.5.31
./configure --sbin-path=/usr/local/sbin --with-http_ssl_module --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_stub_status_module --with-mail --with-mail_ssl_module --with-cc-opt="-I /usr/include/pcre"
make
sudo make install
|
Configuration summary
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
+ threads are not used
+ using system PCRE library
+ using system OpenSSL library
+ md5 library is not used
+ sha1 library is not used
+ using system zlib library
nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/sbin"
nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
nginx pid file: "/usr/local/nginx/logs/nginx.pid"
nginx error log file: "/usr/local/nginx/logs/error.log"
nginx http access log file: "/usr/local/nginx/logs/access.log"
nginx http client request body temporary files: "/usr/local/nginx/client_body_temp"
nginx http proxy temporary files: "/usr/local/nginx/proxy_temp"
nginx http fastcgi temporary files: "/usr/local/nginx/fastcgi_temp"</code>
|
Nginx configuration (updated 09/05/2007)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
|
worker_processes 2;
error_log current_path/log/nginx.error.log debug;
pid shared_path/pids/nginx.pid;
events {
worker_connections 1024;
}
http {
include /usr/local/nginx/conf/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log current_path/log/nginx.access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 70;
gzip on;
gzip_min_length 1000;
gzip_buffers 4 8k;
gzip_comp_level 9;
gzip_proxied any;
gzip_types application/xml application/javascript application/x-javascript application/atom+xml application/rss+xml;
gzip_types text/css text/html text/javascript text/js text/plain text/xml;
upstream mongrel {
server 127.0.0.1:8000;
server 127.0.0.1:8001;
}
server {
listen 80;
server_name fucema.net www.fucema.net;
root current_path/public;
index index.html index.htm;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host "www.fucema.net";
proxy_redirect false;
proxy_max_temp_file_size 0;
# rewrite 'fucema.net' -> 'www.fucema.net'
if ($host = 'fucema.net' ) {
rewrite ^/(.*)$ http://www.fucema.net/$1 permanent;
}
# if static file exists serve now, skip rewrite rules
if (-f $request_filename) {
expires max;
break;
}
# redirect feed requests to feedburner, unless its the feedburner agent
if ($http_user_agent !~ FeedBurner) {
rewrite ^/feed/atom.xml$ http://feeds.feedburner.com/fucema;
}
if (-f $request_filename/index.html) {
expires 7d;
rewrite (.*) $1/index.html break;
}
# support rails page caching
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
# pass it onto upstream mongrel cluster
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}
}
location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|mov).*?$ {
root current_path/public;
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root current_path/public;
}
}
}
|
Mongrel cluster configuration
1
2
3
4
5
6
7
8
9
|
---
cwd: current_path
log_file: log/mongrel.log
environment: production
address: 127.0.0.1
pid_file: tmp/pids/mongrel.pid
port: "8000"
servers: 2
|
