The Codebelay Blog

Author: barce

  • Amazon EC2 in the Enterprise

    This is just a quick summary of what it was like implementing Amazon’s EC2 in an enterprise environment.

    1. You’ll need to write your own LDAP plug-ins to interface with any access control lists. E.G. where I work WordPress is used for corporate communications so an LDAP plug-in had to be written to make sure the right people saw the right information.

    2. Migration can be expensive if you’re using EBS on the first go. On windows, and I’m not sure why, it can cost about $50 to migrate 2GB of data into EBS. In linux, it happens at a fraction of that cost and as advertised.

    3. Windows can be very expensive. Although they say it’s 12 cents per hour per small instance beware of hidden costs like authentication services and SQL server. With both, you are using a server at the cost of $1.35 / hour, which IMHO could be run cheaper with just a small linux instance and do the same thing at 10 cents per hour.

    I’m pretty sure that with the right Amazon EC2 set up you could run a cluster of servers for a Fortune 500 company for under $1000.00 (one thousand dollars) per month without the CapEX costs associated with new hardware.

    If you have any more questions about Amazon EC2 in the enterprise I’d be happy to answer them. Please ask them in the comments below.

  • How to Load Balance and Auto Scale with Amazon’s EC2

    This blog post is a quick introduction to load balancing and auto scaling on with Amazon’s EC2.

    I was kinda amazed about how easy it was.

    Prelims: Download the load balancer API software, auto scaling software, and cloud watch software. You can get all three at a download page on Amazon.

    Let’s load balancer two servers.

    elb-create-lb lb-example --headers \
--listener "lb-port=80,instance-port=80,protocol=http" \
--availability-zones us-east-1a

    The above creates a load balancer called “lb-example,” and will load balance traffic on port 80, i.e. the web pages that you serve.

    To attach specific servers to the load balancer you just type:

    elb-register-instances-with-lb lb-example --headers \
--instances i-example,i-example2

    where i-example and i-example2 are the instance id’s of the servers you want added to the load balancer.

    You’ll also want to monitor the health of the load balanced servers, so please add a health check:

    elb-configure-healthcheck lb-example --headers \
--target "HTTP:80/index.html" --interval 30 --timeout 3 \
--unhealthy-threshold 2 --healthy-threshold 2

    Now let’s set up autoscaling:

    as-create-launch-config example3autoscale --image-id ami-mydefaultami \
--instance-type m1.small
    as-create-auto-scaling-group example3autoscalegroup  \
--launch-configuration example3autoscale \
--availability-zones us-east-1a \
--min-size 2 --max-size 20 \
--load-balancers lb-example
    as-create-or-update-trigger example3trigger \
--auto-scaling-group example3autoscalegroup --namespace "AWS/EC2" \
--measure CPUUtlization --statistic Average \
--dimensions "AutoScalingGroupName=example3autoscalegroup" \
--period 60 --lower-threshold 20 --upper-threshold 40 \
--lower-breach-increment=-1 --upper-breach-increment 1 \
--breach-duration 120

    With the 3 commands above I’ve created an auto-scaling scenario where a new server is spawned and added to the load balancer every two minutes if the CPU Utilization is above 20% for more than 1 minute.

    Ideally you want to set –lower-threshold to something high like 70 and –upper-threshold to 90, but I set both to 20 and 40 respectively just to be able to test.

    I tested using siege.

    Caveats: the auto-termination part is buggy, or simply didn’t work. As the load went down, the number of the server on-line remained the same. Anybody have thoughts on this?

    What does auto-scaling and load balancing in the cloud mean? Well, the total cost of ownership for scalable, enterprise infrastructure just went down by lots. It also means that IT departments can just hire a cloud expert and deploy solutions from a single laptop instead of having to figure out the cost for hardware load balancers and physical servers.

    The age of Just-In-Time IT just got ushered in with auto-scaling and load balancing in the cloud.

  • Monitoring Websites on the Cheap: Screen and Sitebeagle

    If you don’t fail fast enough, you’re on the slow road to success.

    One idea that I recently failed was using a screen and sitebeagle to monitor sites.

    It’s not a complete failure… it works okay.

    Due to budget constraints, I put my screen and sitebeagle set up on a production server.

    For some reason that production server ran out of space and became unresponsive. Screen no doubt caused this. I was alerted of the issue and did a reboot.

    After the reboot, although Amazon’s monitoring tools told me the server was okay, the server was not. The MySQL database was in an EBS volume and needed to be re-mounted.

    The solution I now have in place is still screen and sitebeagle. But I use another server with screen and sitebeagle on it to monitor the production server that gave me the issue in the first place.

    It’s a question of who will monitor the monitors… in a world of web sites with few site users the answers pretty bleak. In the world of super popular commercial sites, the answer’s clear. The wisdom of crowds will monitor the web sites.

  • Lone Coder in a Sea of Power Users?

    Hey folks, I might expand this into a larger article for either mashable.com or techcrunch.com .

    I was wondering if you folks who are coders feel that you’ve been put in a situation where you are the lone coder in a sea of power users?

    If so, is this situation ideal for you? Not ideal?

    How do you deal with job queues?

    How do you deal with working with power users with conflicting interests?

    I’m really interested in war stories where you feel you’re the lone expert.

    Cheers, Barce

  • Git: How to Cherry Pick Commits and Package them Under a Tag

    I’ve pretty much come to rely on git to pull me out of any bad jams in the chaotic environment I work in.

    One thing I’ve had to learn to do is cherry pick commits and package them under a tag in git.

    Here’s how to do it if you were working with my newLISP project called Sitebeagle:

    fork sitebeagle on this page

    cd sitebeagle

    git fetch –tags

    git checkout 8f5bb33a771f7811d21b8c96cec67c28818de076

    git checkout -b sample_cherry_pick

    git cherry-pick 22aab7

    git cherry-pick b1334775

    git diff sample_cherry_pick..master

    git tag leaving_out_one_commit

    git push origin –tags

    At this point, you should have a tagged branch that doesn’t have the commit with the change to the “2nd file.” The diff should look exactly like this:

    diff –git a/test.lsp b/test.lsp
    index 9cf1667..158b625 100755
    — a/test.lsp
    +++ b/test.lsp
    @@ -1,6 +1,7 @@
    #!/usr/bin/newlisp

    ; test tag test_a
    +; cherry pick test 2

    (load “sitebeagle.lsp”)
    (load “twitter.lsp”);

  • A Cross Platform Browser, Windows 2003 EC2 AMI

    I recently created a cross platform browser, Windows 2003 EC2 AMI: ami-69739500

    It has the following pre-installed:

    • gvim
    • IE 7
    • Firefox 3 with Web Developer, yslow & Firebug
    • opera
    • Putty SSH
    • Putty SCP

    Pretty much with that list you’re all set to do troubleshooting for cross platform browser issues.

    There’s IIS 6.0 and SQL Server, too.

    I’ve linked the password to this ami at http://www.codebelay.com/ami-69739500.txt . It’s a short-coming of Windows AMIs on EC2 that I have to link the password, so please change it once you get into the instance.

  • Cylons will be built with Open Source Software

    It looks like Open Source will contribute to the building of Cybernetic Life Nodes.
    These open source projects are leading the way:

    The only thing missing is a robotic body to put this stuff in.

     

    It's not easy being virtual.
    It's not easy being virtual.

  • Testing the iPhone WordPress App

    Just typing this from my iPhone. It works really sweet & is full of win.

  • No Facebook or Twitter for a Week

    What could go wrong?

    The Bad:

    1. Work now requires me to put into my timesheet how long I take for lunch.
    2. I missed an appointment on Thursday that was in my facebook events
    3. I missed wishing @angmoore Happy Birthday on her Birthday. Happy Birthday, Ang!
    4. I missed the Flickr Party which was in my facebook events
    5. There’s a lot of side work that I missed from search.twitter.com It’s really a great tool for finding work.
    6. I explained to some folks why I hadn’t approved their friend requests or follows in person. That led to interesting conversations.

    The Good:

    1. I felt a lot more focused
    2. There is a shared illusion of connectedness that can prevent you from achieving your dreams. I won’t let that happen to me.
    3. I avoided making any embarrassing statements, grammar mistakes or spelling mistakes
    4. I enjoyed really focusing on people physically close to me.

    Nuggets of Wisdom:

    1. I didn’t miss twittering unless I was at a party or bar where I didn’t know anyone.
    2. Twitter’s search is Foucault’s panopticon.
    3. It’s what you don’t do on Facebook that gets you hired.

  • Gear6.com Hijacks @memcached on Twitter

    By taking the @memcached username on Twitter, Gear6.com has engaged in identity theft. Yes, someone call the police now.

    I’m pretty sure this is one of those posts that won’t make anyone happy.

    Let me first start off with what is meant by identity theft. Wikipedia provides a pretty solid, working definition. If you disagree with it, then I’m curious as to why and how, but for this discussion, I’m using it to prove a point.

    “Identity theft is a crime used to refer to fraud that involves someone pretending to be someone else in order to steal money or get other benefits.”

    For example, if I walked around telling people I’m Chow Yun Fat while flashing a fake ID that says Chow Yun Fat, and I say, “Let me into this party. Wasn’t Crouching Tiger Hidden Dragon an awesome film?” then I’m stealing Chow Yun Fat’s identity. There is no way he’d say, “It’s cool, go ahead and pretend to be me.”

    Let’s take a look at @memcached on Twitter.

    First off, having the url, http://twitter.com/memcached , is like having a business card. A business card cannot be used to legally establish identity, but it strongly suggests an identity.

    In this case, I would expect that http://twitter.com/memcached is someone, or a group of persons working on the memcached open source project.

    Let’s say I go to the url for the memcached twitter account.

    What do I see?

    http://twitter.com/memcached/status/1376487185

    @bdeshong How are you liking the new changes in 1.3.2? Digging the new stats?

    I see a few posts of @memcached pretending to be a member of the memcache open source coding team just like some dude pretending to be Chow Yun Fat at a party.

    And here’s bdeshong thinking he’s talking to memcache:

    @memcached Haven’t had a chance to spend any time with it just yet. In the next week or two, though…

    http://twitter.com/bdeshong/status/1376501081

    There are other examples where they announce releases of memcache as if they were the memcached development team.

    This is the equivalent of me signing up with the @twitter account and announcing based on the twitter blog that twitter has released new code.

    Now they’re clearly pretending to be someone else, but are they doing it for stealing money? Getting other benefits? Well, no money has changed hands here. There are just a lot of confused people, but confusing folks isn’t a crime.

    Are they getting benefits?

    Yes! In the Internet age attention equals money in terms of pageviews. They are definitely getting the benefit of the advertising garnered through the memcached name!

    This sort of reverse-swiftboating where the stolen, good name of an open source software project is used to prop up business interests. I really do believe that building scalable websites is a good business model, but please don’t steal names that represent reputations you haven’t earned.

    I ask @ev and the folks at Twitter to please take away @memcached from Gear6.com and give it to the *real* memcached community.