Thursday, August 18, 2016

Quick 1..2 on how to create an EMR cluster with a custom boostrap action (BA)

This is more a reference for me than for those reading this. I know I'm going to have to do it again in the future and I really don't want to run into the problems I had in getting this to work.

So, the AWS cli command to create the cluster:

myBA='--bootstrap-action Path="s3://< my-s3-bucket >/shell/install_profile.sh"'; aws emr create-cluster --release-label emr-5.0.0 --name testBA-10 --applications {Name="Hive",Name="Spark",Name="Zeppelin",Name="Ganglia"} --ec2-attributes my-Keypair --region us-east-1 --use-default-roles --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m3.xlarge InstanceGroupType=CORE,InstanceCount=1,InstanceType=m3.xlarge ${myBA}

The environmental variable myBA is the path to the S3 bucket that stores my script for the BA. It's worth noting that the script should exit with a 0 exit value if you don't want your BA to fail. Here's my script: #!/usr/bin/env bash set -x set -e HADOOP_HOME=/home/hadoop # My profile aws s3 cp s3://< my S3 bucket >/AWSStuff/std/myfuncs ${HADOOP_HOME} echo "About to modify my .bashrc login" echo -e "\n. ~/myfuncs\ngetNewProfile\n. .hbw" >> ${HADOOP_HOME}/.bashrc # Install byobu sudo /usr/bin/yum -y --enablerepo=epel install byobu # And we're ready to rock exit 0 Bingo. We're ready to roll.

Sunday, July 17, 2016

s3-dist-cp and regex's and the groupBy switch


I spent an inordinate amount of time trying to work out how s3-dist-cp [and more specifically, the  groupBy flag] works, because you would think that the reference to the Wikipeda link on regex's would mean that the Perl compatible regular expressions would work.

Not so fast there Tonto.....

It took me a good deal of experimenting to find this out. Hopefully this blog post helps you save some time....Please post your saved time to me at the address at the bottom of this post!

test-split/split-1/full
                  /full.1
                  /full.10
                  /full.11
                  /full.12
test-split/split-2/full.14
                  /full.15
                  /full.16
                  /full.17
                  /full.18
test-split/split-3/full.2
                  /full.20
                  /full.21
                  /full.22
                  /full.23
test-split/split-4/full.25
                  /full.26
                  /full.27
                  /full.28
                  /full.3
test-split/split-5/full.5
                  /full.6
                  /full.7
                  /full.8
                  /full.9

Operators within the '()' will be 'saved' for the output.

I tried this regex:
--groupBy '(full).1[5-7]'

expecting that it would groupBy
full.15, full.16 and full.17

Nope, it didn't work until I did this:
--groupBy '.*(full).1[5-7]*'

So it seems the regex is looking for anything (.*) prior to 'full' too - the full path!

The --groupBy is not a RE as I was expecting it to be!

Once figuring out that rexex's are not really regex's, I tried this:

s3-dist-cp --src s3://s3-lab/test-split/ --srcPattern='.*split-[12].*' --dest hdfs:///user/hadoop/PBXfull/ --groupBy ".*(full).1[0-9]*"

In the above, I expected s3-dist-cp would use a source pattern of .*split-[12].* for the directories and then the groupBy to select the files WITHIN those directories. Something was wrong though, because output ended up [seemingly] randomly in split-2 or split-1. I renamed the directories to foo, baz, bar, ipsum, lorum as this was my use-case.....

test-split/foo/full
                  /full.1
                  /full.10
                  /full.11
                  /full.12
test-split/bar/full.14
                  /full.15
                  /full.16
                  /full.17
                  /full.18
test-split/baz/full.2
                  /full.20
                  /full.21
                  /full.22
                  /full.23
test-split/ipsum/full.25
                  /full.26
                  /full.27
                  /full.28
                  /full.3
test-split/lorum/full.5
                  /full.6
                  /full.7
                  /full.8
                  /full.9

Then I ran this:

s3-dist-cp --src s3://s3-lab/test-split/ --srcPattern='.*(foo|bar|baz).*' --dest hdfs:///user/hadoop/PBXfull/s2/ --groupBy ".*(full).1[0-9]*"
     
This DIDN'T do what I expected (I expected it to grab foo, baz and bar, sifting out only the files full.10, full.11, etc).

So I tried this instead:

s3-dist-cp --src s3://s3-lab/test-split/ --srcPattern='.*baz.*' --dest hdfs:///user/hadoop/PBXfull/s2/ --groupBy ".*(full).1[0-9]*"

And now it worked. The main problem is that the srcPattern means that the output is in PBXfull/s2/baz while in the (foo|bar|baz) it put all the output into PBXfull/s2/foo....

But it doesn't end there....
What I would really like is that all the files in foo be concatenated, then all the files in baz, then all the files in bar, etc. Not one large file.

Thanks for Barbaros who got the solution and I'm documenting it here.

So in trying to work this out, I tried:

   s3-dist-cp --src=s3://s3-lab/test-split/foo/ --dest=hdfs:///user/hadoop/PBXfull/ --groupBy=".*(full)\..*"

This joined the files in test-split/foo together into a file 'full', but clearly this is going to break if I try to join the files in bar, bar, etc...

Let's test a couple of options:

   s3-dist-cp --src=s3://s3-lab/test-split/ --dest=hdfs:///user/hadoop/PBXfull/ --groupBy=".*(full)\..*"

Yup. It creates a file (full) combining all the files and outputs it to one (presumably the last) directory (in my case, I still have split-5 knocking around).

   s3-dist-cp --src=s3://s3-lab/test-split/ --srcPattern=".*bar.*" --dest=hdfs:///user/hadoop/PBXfull/ --groupBy=".*(full)\..*"

Yup. This does what I want. bar is concatenated into a file full in PBXfull/bar/full, but there's still a problem. The main one is that I have 10000 of these foo, bar, baz directories and I really want evey one in it's own directory, so I can't specify the srcPattern every time. It'll drive me crazy....crazier...

In the end, I tested Barebaros' solution:

   s3-dist-cp --src=s3://s3-lab/test-split/ --dest=hdfs:///user/hadoop/PBXfull/ --groupBy=".*/(\\w+)/.*"

The groupBy here won't group by the filename full.*, but WILL group by the diretory foo, bar, baz, etc....i.e. all the full.* files will be concatenated and grouped by the directory.
 
This produced:
   PBXfull/foo/foo [note the filename is now foo, not full]
   PBXfull/bar/bar [again, the filename is bar, not full]
etc.

The groupBy here is grouping by the directory and not the filename!

Wow! So nothing is quite what it appears to be.

A final example:
  s3-dist-cp --src=s3://s3-lab/test-split/ --dest=hdfs:///user/hadoop/PBXfull/ --groupBy=".*/(split)\-([0-9])/.*(full)\.1[0-9]*"

This, in theory groups by the split-1, split-2, etc. Or not. The output is:

   PBXfull/split-2/split2full

And this makes sense because:
   the first part of the groupBy (split)\-([0-9])
   will consider the split-1, split-2, etc.
And the second part .*(full)\.1[0-9]*
  will only consider files:
     full.10, full.11, full.12, etc.

And there are only 2 directories with these files in them, namely split-1, and split-2. Which means the last 'split-' will be where the output of the files goes - hence split-2/split2full.

Ok. Don't worry to send me the time you spent reading this post :-), but hopefully it helps someone (perhaps me even) sometime in the future.

Wednesday, May 25, 2016

Loading CSV data into DynamoDB using Hive on EMR

To complete this recipe, you will need:

* An AWS account and an EMR cluster with Hive installed (it's the default when launching a cluster 4.6 EMR cluster)
* Some data in CSV format that you can load into an S3 bucket (2)
* A DynamoDB table into which you plan to load this data (note, you need not specify all the column names - just the primary key and perhaps extra hash keys)

The data I'm going to use is a set of data that looks as follows:
   id; fortune; randomNumber

I generated this using the fortune(1) program on Linux. The random number from $RANDOM in bash and of course a count.

   1;Q:Why did the germ cross the microscope? A:To get to the other slide.;22913
   2;Do not sleep in a eucalyptus tree tonight.;2448
   ...

The general gist of this is to load the data into S3, then create a meta-table in Hive that will allow Hive to "select" through this meta-table from S3 and "insert" into the DynamoDB table.

Upload the data to s3 using the AWS cli:

 $ aws s3 cp fortunes.csv s3://*my-s3-bucket*/mydata/
  
Once you've created the EMR cluster, log into the master node using ssh:

  ssh hadoop@ec2-52-3-247-254.compute-1.amazonaws.com

Now it's time to create your meta-table in Hive for the S3 data. Type

  $ hive

(to get into the hive prompt)
 
  hive> CREATE EXTERNAL TABLE fortunes_from_s3 (id bigint, fortune string, randomNum bigint) ROW FORMAT DELIMITED FIELDS TERMINATED BY "\;" LOCATION 's3://*my-s3-bucket*/mydata';  

At this point, you should be able to "select" from this meta-table. This is a good test:
  select * from fortunes_from_s3;

If you get results that you expect from your data in S3, then we're half-way there.

Now it's time to create an external meta-table (hive_test_table) in DynamoDB to load the data.

  hive> CREATE EXTERNAL TABLE hive_test_table (ID bigint, Fortune string, RandomNum bigint) STORED BY 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler' TBLPROPERTIES ("dynamodb.table.name" = "", "dynamodb.column.mapping" = "ID:id,Fortune:fortune,RandomNum:randomnum"); (1)


Note that one needs to map the fields, in this case:
  ID maps to id (in DDB), Fortune maps to fortune (in DDB), etc., where ID is the Hive meta-table column, Fortune is the Hive meta-table column.


We've got to the last stage to do at this point; loading the data into DynamoDB. To do this, one "inserts" from the "select":

  hive> INSERT OVERWRITE TABLE 'hive_test_table' SELECT * FROM 'fortunes_from_s3';

There will be a load of stuff that'll fly by as it loads the data:

  Query ID = hadoop_20160525155656_2ad0b238-f350-4805-8d8a-ebe9646f276c
  Total jobs = 1
  Launching Job 1 out of 1
  Number of reduce tasks is set to 0 since there's no reduce operator
  Starting Job = job_1463217542741_0016, Tracking URL = http://ip-192-168-0-122.ec2.internal:20888/proxy/application_1463217542741_0016/
  Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_1463217542741_0016
  Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
  2016-05-25 15:56:34,246 Stage-0 map = 0%,  reduce = 0%
  2016-05-25 15:56:51,858 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 4.31 sec
  MapReduce Total cumulative CPU time: 4 seconds 310 msec
  Ended Job = job_1463217542741_0016
  MapReduce Jobs Launched:
  Stage-Stage-0: Map: 1   Cumulative CPU: 4.31 sec   HDFS Read: 200 HDFS Write: 0   SUCCESS
  Total MapReduce CPU Time Spent: 4 seconds 310 msec
  OK
  Time taken: 32.273 seconds



It's time to look in DynamoDB to check whether the data are loaded. If they're there, then the job is done.

Permissions

IAM may well stop you doing this job. In order to make sure the data can be loaded, create a new role in IAM. This role should have, at minimum the following Policies attached:

  AmazonElasiticMapReduceRole
  AmazonElasiticMapReducefor EC2Role


This will ensure that EMR can get the data from S3. Attach this role to the DynamoDB under Access Control policies for the table.

Gothas

(1) The dynamodb.column.mapping can catch one out. The ID:id,.... should not have space between the various column mappings (i.e. after the "," and before the next column mapping) 
(2) make sure that the CSV is the only thing in the bucket and don't specify the actual file name. Hive will try to read the file directly from the bucket  

Tuesday, May 1, 2012

Human resources vs Personnel

Human Resources vs Personnel department

So something DaveK said to me the other day got me thinking about this. His comment went something like this "the problem with HR was when they changed it's name from personnel department to HR". I thought he was making too much of a fuss really, in that they were both pretty ineffective.

But something in me has made me ponder this point for longer and the more I think about it, the more I realise that calling someone a "human resource" is a bit like calling someone a gogga, or some other form of un-PC hate-speech. What was it in the apartheid days - dehumanise them and then it's easier to persecute them. So now here we are, human resources. We're not people any longer. We're just resources that, well are human, but can be thought of primarily as resources. And what's a resource anyhow? Mirriam Webster describes a resource as "a source of information or expertise". So if humans are resources, and we apply this definition then one would expect, being a source of information and expertise would mean that the humans in question get treated with some degree of value.


My experience of HR though, [ and not only in my present position, but in previous companies ], is that the humans which the HR department should be valuing, are seen as of little value. IT used to have [ and in many places still does ] a reputation of being out for themselves. They couldn't care a hoot for the user or what they want and need. My experience is that HR have the same selfish attitude. So why is this the case? Why is it that HR people seem to wield such power in organisations and seem oblivious to the very people they should be serving who are of value to the organisation?


"Great Human Resource professionals add value to any organization. Recruiting and retaining star performers, building a productive workforce, coaching managers to perform at higher levels, ensuring that the organization stays compliant, and raising the bar on performance are what HR brings to the table"

This may be true, but I've never seen it. Are we just extremely short of great HR people or have the HR people I've ever dealt with lost touch with what's important - the people. So that brings me to the personnel department. Somehow, this keeps the people who should be dealing with people understanding that they're dealing with people. It could just be me, but would a return to the name personnel department make a difference to how they deal with the valuable people in the organisaion?

Sunday, January 3, 2010

Paperless Geocaching with my Nokia e71

Recently I've got into geocaching. Not that I've not been interested before, but I simply didn't want to lay out the heaps of cash they want for GPS's. But, thanks to cell phone technological improvements and an upgrade (not to an HTC with Virgin Mobile!), I have a GPS receiver on my phone.

This started me thinking about doing some Geocaching. This along with the fact that the chillens don't like to go for a walk like my wife and I, and need some form of carrot to get them to walk regularly. Enter geocaching.

This is a tutorial on how to do paperless geocaching primarily because I don't want to have to pre-prepare for this past-time. I want to simply do one if I have the time.

I run Linux, which can sometimes make things a little more challenging. Fortunately I'm not the first to attempt this, but after scouring the 'Net and with some help from a colleague, I've got this right now. So, here are my experiences and technical insights.

What you'll need:
Hardware:
  • A Symbian S60 phone (mine is a Nokia e71) with built-in GPS receiver.
  • A PC that can run ruby (pretty much all machines, but if you can't apt-get it, then you'll need to do some more reading!)
Software:
  • A program called geotoad (Geotoad) (the reason for this will be explained below)
  • SmartGPX (SmartGPX)
Account:
Co-ordinates of where you wish to start from.

Recipe:
Unless you decide to become a premium member of geocaching.com, obtaining caches in the GPX format (which is essentially an XML file of all the caches you select, their difficulty, description, the clue, and the last 5 logs), you're stuffed and can't do paperless at all. This is where geotoad comes in. Geotoad is a neat ruby program that will allow you to define which geocache's you're interested in and then will download them as GPX files. While it has a command line interface (which some may not like), I think it's just grand as it consumes little memory/resources and is quick an easy to use.

To use it, you're going to have to give it your geocaching.com account details. You've signed up an account already. If not, visit Geocaching.com and do so now.

Also, using Google Maps, you can install a GPS co-ordinate plugin, which will allow you to get the GPS co-ordinates of pretty much any place you wish to start your search (filter) from.

Now it's time to fire up geotoad. You'll use options (1) username, (2) search type (can be coords, or other type), (2) the co-ordinates from where you're likely to start your geocaching adventures, and (4) the distance (in miles sadly) as a radius from which you wish to do paperless geocaching.

I personally chose the same co-ordinates and then varied the distance from 50 miles to 200 miles from this point and saved all those in separate GPX files (option 23). In addition, I varied the starting points, so now whether I'm going to the Cedarberg or the Garden Route, I can load up cache's in those areas. Nice!

Once you have set these settings (1-4 and 23), hit (s) for search and then go and make coffee. Geotoad, in order not to put too much load on the servers, backs off while downloading the individual caches. Depending on how many you're loading, it may take up to an hour.

Have GPX file...will geocache!

Not yet big daddy. Time to get some software for the phone. The phone software comes in the form of SmartGPX. It's a great piece of programming and it's not crashed once! Well done guys :-).

You'll need 2 pieces of software actually. The SmartGPX and the AddOns. Download both. You'll have to sign these using the Symbian Signed website (SymbianSigned). For this you're going to need your IMEI number on the phone. Easy if your phone responds to the *#60# keypad code....but mine didn't. Had to remove the battery. Upload both pieces of s/w to the webpage (one at a time) and then download them again. They'll be signed. Install both.

Now it's time to load the GPX file(s). Sterreman (waiting for a link from him) refers to not trashing your speed camera GPX files by overwriting them with this GPX file. I have no experience of this, so if this is a concern to you, do some more research. I can't see it being more complicated than joining the 2 GPX files (perhaps removing some header information from the 2nd one) since the GPX file is only XML and human readable.

Right. Start SmartGPX and Import the GPX file for the place(s) you're likely to be (you've transferred the GPX output from the PC running geotoad to the phone already have you?).

Now, the annoying thing about the inbuilt GPS is that it cannot change the way it's co-ordinates are displayed. They're always in degrees, minutes, seconds and hundredth of a second. Not of any use for geocaching since all the co-ordinates on the site are in degrees, minutes and hundredth of a minute. So, the navigation tool of the Nokia can't understand the GPX co-ordinates. Annoying, I know, but that's where the SmartGPXAddOn's comes in. It allows you to select a particular geocache, and then import those co-ordinates into the 'Navigation tool'. From there, it's simply a matter of using the navigation tool to find the cache.

But wait...there's more.

Once you've found the cache, you can log it on your phone immediately. Nice! There's a 'Log your visit'. So, you can log a note, the time you found the cache, and of course what you took/left. That log is saved on the phone, but clearly, that's a problem if you do multiple caches in a day. You don't want to re-type what you typed into the phone when you actually logged it.

So, when you get back to your computer, you can Export these logs, and import them straight into geocaching.com using this link (Upload Field Notes). That won't publish them immediately but will allow you to modify them before publishing to the website. So....geocaching got easier.

Have fun being the search engine!

Tuesday, November 3, 2009

Telkom monopoly - their dominance continues (to break the rest of us)!

Telkom once again got bad press on the issue of the link between Sutherland (SALT) and the Observatory.


One would have thought, given the competitiveness of business, that Telkom would not wish such bad press on themselves. As I suspect though, they simply couldn't care! In effect, Telkom have responded to the stink these articles created by simply sticking to their guns and demanding over twice the original, signed, contract. They're wanting in excess of R20M for this link - the signed contract was for R10M.

It's estimated that to lay a new fibre, the cost would be approximately R40M, only ~40% more than Telkom are asking. How is this possible Telkom?

1) How do you explain this change in price,
2) the fact that this bad press seems to be ignored by you,
3) how any of us get to continue to do business by excluding Telkom?

Telkom have been embarrassed more than once. The Winston pigeon shows up Telkom for providing a less than broadband solution. They face a penalty for anti-competative behaviour, and yet, they continue to stifle innovation, science, education and the progress of research and business in South Africa. Where will it end?

I cannot tell you how frustrating this is. Apart from anything, Telkom hold the entire South Africa to ransom. As a colleague put it, this makes a mockery of their slogan 'Touching Tomorrow'. One is certainly left wondering whether we'll even be able to compete with other African countries as they speedily de-regulate their telecomms industries

Officials at Telkom this we 'just want the SALT problem to go away' and yet they've dug their heels in saying 'a mistake was made in the tender process'. From what I understand, this is not the only place they're reneg-ing on their deals. So, how does one hold them to their contractual agreements? That a mistake was made is either:
  • Incompetence in drawing up the response to the tender or
  • they're simply lying.
Given the current climate in South Africa, I suspect both!

This morning's Business Report headlines with an article "This decade is marked by march of mediocrity", and oh how we're seeing that in South Africa! Without getting side-tracked into why we've ended up in mediocrity, I do feel depressed about being held to ransom by Telkom; or perhaps now it's more important than ever to stand up and be counted because the 'better life for all' is slipping through our fingers - all because of a lack of decisive leadership.

Friday, July 10, 2009

Meeting people


I guess there are those people who are in the limelight (like let's say, Michael Jackson) who you just know aren't going to be that great to meet 1-on-1. Watching Wimbeldon the other day, I was rooting for Rodger Federer for a number of reasons, but primarily because I think that if I caught him in an airport, he'd give me the time of day.

Today was my last day of 2 weeks looking after my 3 (or 4 or 5) children. It's been a rewarding, sometimes trying but generally interesting 2 weeks. Having 3 certainly helps! But that's not the point of this musing. Cara and I have been trying to build a xylophone for her school project. It was an ambitious project (though at the outset, I thought "how hard can this be!"). We're almost done now. The Internet, some planning on paper and then Cara and I tackled it. There was much sawing and glueing, and we got 90% of the way there. I'd made the keys, but not tuned them. Save my two earbones (and Jenny says those don't work very well; especially when it comes to 'have you washed the car lately?'), I have not a musical bone in my body and the Internet was either overly mathematical or generally vague on how to tune these things. So, true to dad fashion, I phoned a friend. Well; not actually a friend, but someone who makes their living playing the Marimbas.

Enter Ross Johnson.

So, having looked them up on Google, I got hold of amaAmbush productions. The guy I spoke to said Ross would call me back. I thought "ja, right!". But he called and he offered to come to their factory and help me tune these beasties. We duly arrived and what an experience! Ross went out of his way to include the children. He explained what we were about to do, played a piece ('In the Jungle') on one of his Marimbas with them playing along, and focused on their task, including them all the way.


It's funny. For such a long time I've been to see his concerts and enjoyed every minute. He looked like the Rodger Federer type; but who's to know? Yet today, Ross showed why he's destined for great things. It was such a pleasure meeting someone of his caliber. If I were only that influential at his age! Sigh.

I guess he'll not remember me at future concerts, but I'll still believe that he's a really fantastic guy who is unbelievably talented and doing more good here in our funny South African than most other people out there. So there you go.

Tuesday, January 13, 2009

Dwesa marine nature reserve, Wild Coast

Well, we're back in 'civilsation' and boy, it feels so.....primitive; to wake up to the alarm rather than the sound of the Mangrove Kingfisher, or the incessant sound of the cicidas at 4:30am.

I spent so long planning and researching this trip that it's almost a sin to keep this information to myself. A double-edged sword really as publishing it will no doubt encourage more people to visit Dwesa, and the pristine beauty and lack of people will be lost. To those undeterred by lack of electricity, no radio signal (or TV for that matter), minimal cell phone coverage and a long drive to or from Dwesa, this is the perfect spot. This blog is mostly for those who, like me, are finding it difficult to find information about Dwesa. But let's keep Dwesa the beautiful place it is.

The route to get there
Someone I talked to just days before leaving said "Don't take the shortest route, it's not always the quickest", and they were right. We approached from Dutywa side. There's 30km of tar to Willowvale, but that's where the joy ends. Everything is relative I guess, so if you've travelled to Malawi and back in a Huyndai Elantra then these roads are really quite good. Don't expect to do more than 30km/h though. Folks told us that they'd done it at 80km/h - but they also said they'd lost about a half-dozen beers and other less important stuff too (like food). There are a whole bunch of junctions getting there, but sticking to the main drag will get you to the gate just fine. Don't try to get in through any but gate 1, all the others are locked.

The facilities
Basic, but all present and correct. Perhaps not quite the same standard you'd expect in the Kruger, but the friendly staff and the beauty of the place make up for the odd cold shower. There are too few toilets to be shared between all campers, but that's not a problem if you make sure you're there out of season. We weren't, and thanks to some not-so-happy-campers, some of the doors to the new ablution block were kicked in so that they could be used (they were locked when we first arrived). In the end, I guess one could get angry, but that's people for you. There is the old ablution block which has toilets, cold showers and basins that are falling off the wall (well, some of them were), but hey, they do the job. There are no lights in the ablutions blocks, so if you want to shower before sun-up or after dark, take your own lighting.

There's no electricity at Dwesa, but if you ask really nicely, the staff will let you put your freezer/fridge or whatever at the office where there is an electrical feed. Did I mention being nice to the staff....they're really lovely people. Gas is only 52km away in Willowvale, but it's pretty unpleasant leaving the camp to go to a typical African town to get a gas refill - and of course it's re-negotiating that road. When I go back, I'll take more paraffin based stuff - it's cheap and available up the road outside the park at the local spaza shop (cycling distance). In fact, we were there so long that the staff offered to take our gas bottles to be refilled at Willowvale for us (they run their chalets on gas too).

Campsites are not numbered, so as the gate person said, "choose your best site". We certainly chose nice ones - though they're all pretty lekker really. The ones on the 'hill' (marked Caravan Sites), catch a bit more of the wind than the lower sites, but from what we were told, the previous year, the high tide combined with heavy rains put the lower sites a meter under water!

As far as the number of site goes - the booking office will tell you there're 20, but in fact they'll book up to 40 sites, so don't expect this to be limited. One can understand because this is the one time of the year when they make their dough, so cut them a bit of slack. Hey, it's NEVER going to get as bad as Nature's Valley where they pack you in like sardines.

Basins for washing your clothes and dishes are non-existent and I must confess to being a little miffed when I could not take a leak in the urinal because some woman (and I'm not being sexist), appropriated the basins in the men's toilets to wash their clothes or dishes. The local people can be paid to do these chores for you, but since we had little to wash, we never used their services. I can't imagine that it was very costly though and, if it means you can spend more time exploring the park, then they could certainly do with the cash.

Supplies
Gas in Willowvale, but the spaza shop up the road does stock some of the basics; people who are vegetarians who stayed for two weeks said the lovely lady at the shop would order in other veges for you if you asked. Cold beer is available up at the village, but don't expect discount prices unless you are fluent in Xhosa. There's nothing inside the park and while firewood is on sale sometimes at the office, they seemed to have run out most of the times I heard about it. Ice is also available in the village, but I would not rely on it; it's mostly sold out.

We wanted to buy fish, but by all accounts Summer is NOT the time to get fish. All the fisherman who stayed in the park came back empty-handed.

Water is quite drinkable from the taps, so don't bother with bottled water.

Things to do
Beach, birdwatching, walks, cycling in the park (see below for the rules), walking on the beach, fishing (outside the boundaries of the park) or just relaxing. In some ways, this was the one downside of the reserve. Officially, you're not allowed outside the 'boom' without a game guide - and that certainly holds true since you can't drive through the locked boom. However that does not stop you walking on the roads or going for a cycle. You need to be warned that there are Buffalo, and they're said to be very aggressive; though we only saw evidence of them and never spotted a single one. They claim to have over 200 in this little reserve. They have 13 rhino's and then a whole lot of Wildebeest, Zebra, Eland and a smattering of croc's. The birds are fantastic though. From our campsite we spotted 7 of the 9 Kingfishers, and then a whole host of others like the Thick-Billed Weaver and the Olive Sunbird - which I might add, has a phenomenal call.

Walks along the beach are great too, and we took a regular stroll to Cobolo mouth. Swimming is limited. One day in the time we were there the surf was so big it washed open the mouth of the river and pushed back up the river to where our campsite was. Swimming is generally somewhat dangerous with strong rip currents almost every day. No surfing or boogie-boarding to speak of and in Summer, the Easterly blows every other day but not strong enough to kitesurf - I think I had 4 kiting session in toto. Life guards are provided for about 10 days between Christmas and New Year, but from what we could see, they probably would not be able to rescue you if you got into trouble.

On this note, we spoke to someone who broke her hand while in Dwesa and they had to drive back to Dutwya to get it cast, so be careful when swimming as there's no NSRI that will come to your rescue. You'll become shark food pretty quickly.

While the boom is closed most of the time, there are 2 days (26th December and 1st January) when the staff open the boom and allow all the residents free roaming in the park. This is the day when the local people converge on the beach for the day and certainly an interesting experience. Sadly there's quite a bit of alcohol abuse over these two days, so we were advised not to get too involved on the beach. While we did go down to take a look at what people were up to, we took their advice and gave the beach a miss on those date, rather heading into the park to explore a bit without the guide.

The guides (Vuyani or Boy-Boy) come at around R45 per adult whether walking or in the car. We used Vuyani quite a bit and he was a very obliging guy who certainly knew his stuff. From what I understand, Boy-Boy is less knowledgeable but really very nice too.

Access to the Haven hotel can be done. Margot, Brenda and Kara did go there one day, but came back without praises. We never bothered. It seemed that the Haven, who have boats to lift one across the Mbashe river were charging R50 per adult to cross.....You could swim, but I guess the current is pretty strong and there are sharks and crocs to contend with (or so were were told).

While walking was limited to taking a guide, we did go on an outing one day where we walked over the headlands. It was a lovely walk, but Vuyani did have a hernia when he heard where we had been. There are rangers who wander all over and they would certainly have chased us back had we run across them.

The weather
As stated, the Easterly is the predominant wind. It did blow strongly some days, but on the whole the campsite is pretty sheltered. This is a summer rainfall area, and judging by the green forest, it's pretty evident that it DOES rain regularly. There was hardly an evening that we didn't have a shower of some sort (light drizzle or a thunderstorm or whatever) - but happily they were mostly short-lived. There were probably two nights when it bucketed down for a couple of hours, but then by morning it had stopped (if not cleared completely). It's warm though, so you can easily walk around without being wet and cold.

Other factors
One of the things I was concerned about was crime. I'm happy to report that crime inside the park is non-existent - except that the monkeys will steal your food if you leave it lying about. Open tents, trailers and cars are an invitation to the Vervets. We took rubber snakes which we dotted around outside our tents and in the trees and that, for the most part did keep monkeys at a safe distance.

Snakes were a factor too. On day 3 or 4 a smallish puff-adder slithered into our tent. Fortunately we spotted it and encouraged it on it's way. It did concern us that there were more about, but we didn't see another for the rest of our stay. We did however spot a very large Boomslang raiding a Weavers nest in the trees above us....but they're harmless to humans unless you go pushing your finger down their throat!

Summary
Would I go back? Absolutely! Was it a great place? You bet. Were the small issues with the facilities a problem? Nope, not for us.

Have a great time if you do go....you may even bump into us there!

Sunday, May 25, 2008

ICTS Service at UCT

Or lack thereof: I marvel how an academic institution that aspires to be a leading African University can tolerate such a bunch of incompetent IT people going under the name ICTS (Information and Communication Technology).

It is not the first time the network has been brought to a standstill by a virus. What's more amazing is that this time, the virus in question was already a couple of weeks (if not months) old. Oh, said a spokesperson who is fairly senior in the ICTS cogs, "McAfee didn't pick it up". No sh*t Shurlock. The virus took my friends computer out for at least a morning. Fortunately I picked it up and cleaned the virus off before it had a noticeable impact on their work.

Almost a week after it was cleaned off the friends computer, they got a mail saying they (ICTS) now knew what the problem was. These people are real rocket scientists! It also took out all the computers in their department; but those people weren't lucky enough to have an IT guy on-hand and had to wait for ICTS to send someone to fix it. So the wait began.

Now get this. Despite paying huge sums to McAfee, no doubt, ICTS responded that the free AVG, was able to detect this virus and that all staff should download and install AVG on their computers to avoid the risk of a re-infection while ICTS mopped up. So, on goes AVG. Then a week later, the affected person asks why AVG updates are not coming through, only to be told that ICTS has blocked AVG updates because they don't want people using it! So now they resort to updating AVG in the evening from their broadband connection.

Network
For the past 2 months, the person in question has complained that on some days, they have no network at work; other days no Internet. But this does not apply to all staff in the same department. Some do, some don't. Then on other days it may be the reverse. They have Internet, others don't.

Log a call
they're told! Great. Here's to another long wait. And I thought Telkom was incompetent. In addition, what with TENET doing the switch from some Telkom service to SANREN or some other provider, Internet is so slow it's not worth using. Some (and I suspect many) have resorted to buying a 3G modem and data bundles which they use at work; off their research grants. That way they've got Internet when they need it. No cr*p with ICTS. Sure, they're not protected by a firewall or have a proxy (which forces them to log in first - sometimes also not working properly), but they can at least get on with their real work. In other cases, departments have multiple ADSL connections and a complex setup of rotating the accounts when they reach their cap.

Email
An IT organisation worth it's salt has now got spam scanning down off-pat. I mean, 1 spam per week is just acceptable. This person gets about 30 to 50 a day. No, they do not sign up their email address all over the show! Like others, this person has resorted to giving out a Gmail email address instead of the official UCT one and is assured of getting less spam that way.

The upshot. They work from home more often now, as they can still get email. They've resorted to get IT support from me (as this stage at no charge) if need be, they don't not have to log onto the network, or put up with ICTS nonsense at every turn. They are thus more productive.

I guess the the question then is: Why is nothing done about ICTS at UCT? Is it because the big cheeses get a 'different' level of service to the rest of the University and are oblivious of this sort of non-service? Perhaps it's because there's some political wrangling with the ICTS department - hey, corruption is not limited to government officials :-)

Finally, if ICTS employees were paid badly, I could half understand. However, I know that ICTS people on the whole are paid more than people in the industry. How do I know this? Because I was put forward for a position with them through an agent. No, I did not take it. And no, I am not bitter. I had to ask myself this question:
"Did I want to be associated with a department/division that is literally the laughing stock of the rest of the university?"

Could I live with myself, getting up in the morning knowing that today I could contribute nothing to the greater good of the organisation?

And perhaps this is the nub of the problem: which competent IT professional would want to tarnish their reputation working for ICTS. The good people that do work there are so dis-empowered to make a difference they loose the will to want to.

In the end, I turned down an interview (which I may add, I only received confirmation of almost a month and a half after my CV was submitted - the HR department at UCT! but that's a story for another blog). I had reservations from the start with the personnel agent, but this has all been confirmed by events surrounding this virus thing.

It's well understood now that having excellent IT facilities is what will make business competitive into the future; but academic progress is just as dependent upon good IT infrastructure. Poor IT services on the other hand leads to organisations being left behind....

VC, DVC and heads of faculties - please take note and do something before this leading African University becomes a dinosaur in our history books.

Xenophobia in SA

Politics We're facing a shameful time in our history of building a new Rainbow Nation. Clearly the rainbow does not encompass all the colours - only those colours that are deemed from South Africa originally. But where does this all start....and more importantly how does it all end?

We have indeed become a mob.....

I stand back and look at the mob tactics used by the followers of one of our leaders at a rape trial in 2006. Little if nothing was done to stop or address that (amazingly it's really hard to find any 'official' press reports about the violence). The infighting prior to the Polokwane conference where a mob mentality prevailed. The ANC Youth League congress where violence erupted; the congress was eventually 'suspended' (at who's cost?) due to the violence that erupted. The continual infighting between Zuma and Mbeki factions. The mob that attacked and stripped naked a woman at a taxi rank for wearing a miniskirt....And now we have mob violence against foreigners. This is most shameful!

How did it all start?

Perhaps it was the fact that Mbeki in his lack of action on the crisis in Zimbabwe has left Zimbabwe to sort out it's own problems. Clearly this is impossible for the people - so they flee to....SA. The land of milk and honey.

Perhaps it's this constant Zuma vs Mbeki feud that seems not to be resolved (there's a lot written on this subject!). If our leaders can't shake their differences for the sake of betterment of the nation, how do we expect that people who have conflict in their townships (really close to home) will be able to handle this conflict. With rising food prices, cost of transport, and repo rate etc. people are feeling the pinch....Every cent they get needs to be counted and the disparity between the have's and the have-nots grows into anger and frustration. Tito - re-read the Harvard Group report and take note!

Perhaps it is the fact that the fat-cats in their bling lifestyle are not interested in the poor - as long as they continue to vote for them in 2009. So they'll kiss up (Zuma seems to be kissing up to everyone who has a brain small enough not to remember his prior kissing-up flow of drivel to a different audience) to them in the run-up - no doubt with a great deal of rhetoric about what they've done for them, or will do for them or how bad the opposition is, etc. etc. (if that fails, they'll resort to dredging up apartheid, race or colonialism). And once the voting is done, we'll have a new bunch of bling-less leaders who want to join their bling-filled compatriots. And will the problems be resolved or even addressed? No. Unlike Mandela who (though he no doubt has his faults) lead his people, our current leaders (and no doubt our incoming leaders) are simply governing the people. A high-and-mightly approach as evidenced by the Mbeki years.

Perhaps it's the fact that after 14 years of democracy, we still see (the majority of) people who are living below the minimum wage or no wage at all. These people probably have lost hope of bettering themselves, and see little or no hope of bettering the lives of their children. Would I not be bitter at the thought of that! Absolutely.

Finally perhaps it's the fact that immigrants who come to our country are ready to work; eager to put in long hours if need be to better themselves. Often they are better educated than local South Africans and thus able to fill positions to satify the employment equity quotas. Even those that don't hold white-collar positions are industrious; making things from wire to sell at the lights, opening shops, getting an education. Sure, there will be a criminal element too, but that goes for every country that has unemployment at the level we have.

Do I condone the mob tactics? Under no circumstances! However I do understand why people are frustrated and the foreigners are just the latest scapegoat.

But how to stop this?

Leaders need to take a stand on this. Groups and churches need to mobilise to protest that the leaders do something, even advertising campaigns by both government and the private sector. Our public leaders extending the hand of friendship publicly to those who are bearing the brunt of the violence. Stop the infighting between our leaders! Do something about the crisis in Zimbabwe, improve border controls to stop the influx of people into SA. Look at alternatives to simply raising the repo rate to curb inflation and the government actively looking at how to reduce unemployment.

And most of all...tackle the mob mentality of resorting to violence to bring about change or to oppose the other guy, or simply to make a point!