Category Archives: Computing

Data Obsessions while in Self-Quarantine

I sit in my home office looking into a garden which explodes in yellow from the forsythia with splashes of pink from the camellias. Both flourish after a large shading cherry tree fell down a few years ago. The tree stump is covered by moss and provides a natural border. My native American Flame azaleas (Rhododendron calendulaceum) now stand 8 feet tall in front after I planted them in 2001 as 3 inch sticks. They are the pride of my garden along with Piedmont, Sweet, Okonee, and Plum azaleas all purchased from Callaway Gardens in Georgia. They grow well, because I correctly predicted that the warmer climate zones of Georgia would move northward towards Delaware. Here are the azaleas in blooms in early May or four weeks from now:

These are distractions, because I need to process and analyze ocean velocity data off Greenland. My student from South Korea rightfully expects numbers that she can work with for her Masters degree. We plan to meet via Zoom video call every Friday and Wednesday. She is ordered to stay at home in Maryland while I am ordered to stay at home in Delaware. We also meet Monday and Wednesday evenings when I teach “Waves” via Zoom to eight University of Delaware graduate students from China, South Korea, Thailand, and the USA. Our topic yesterday was the waves in the wakes of a ship or a duck or an island. To me physics are as beautiful as are the flowers in my garden:

Now these are the things that I should work on during my self-quarantine, but I am obsessed and distracted with new data. The Johns Hopkins University in Baltimore, MD distributes data on the number of people who were diagnosed with Covid-19, who died of it, and who have recovered. While it is easy to access their excellent data displays as global health authorities report them, the actual raw digital data files are accessible at

https://github.com/CSSEGISandData/COVID-19

These data require computer programming and data handling skills that a well trained physical ocean, climate, or data scientist masters. The raw data, however, do not tell a story, because it just looks like gibberish,

but there is a most orderly system to this madness. With 143 lines of computer code (one C-shell and two awk scripts) I convert these data into a single graph to tell a story:

First, I focus only on the number of people who have died, because I consider this the most reliable (albeit morbid and depressing) estimate of how the virus is spreading.

Second, I present the number of people who died relative to the population. It hardly seems fair to compare the numbers from the USA with 327 million people to those of Malta with only 0.5 million people. The technical term is “normalization,” that is, all numbers are relative to 1 Million people. So, 5 dead in Malta give 10 dead per million. The same 10 dead per million correspond to 3270 dead Americans. This way I am comparing apples to apples as opposed to Americans to Maltese.

Third, I want to compare the spread of the pandemic over time on different continents, different countries, different states, and different cities. This requires to time-shift countries hit by the virus earlier than others. In the above graph, for example, I moved the curve for Italy 14-days forward and that of Spain 6-days forward relative to all other places listed.

Fourth, I am most interested in New York State (population 20 million), because it contains New York City (population 8 million) and, I believe, it gives Americans a good idea what is coming. Furthermore, I believe, that the Government of New York State is a little more efficient, smart, and forward-thinking than many other government entities. It also has resources not necessarily available to less affluent communities.

The curve for New York State initially (until Mar.-25) followed the trajectory of Italy 14 days earlier, but then it switched over to the steeper trajectory of Spain 6 days earlier. Notice that Italy’s curve has a flatter trajectory than the steep curve of Spain and New York State. From Mar.-28 to Mar.-31 the New York curve was almost exactly that of Spain 6 days ago, but yesterday, the number of people dying in New York grew even faster than those in Spain or Italy ever did. This is scary stuff.

Yesterday, New York State had about 111 dead per million people. While this is still less than the 180 dead per million people that both Italy and Spain had yesterday, it may take only 4-5 additional days for New York State to reach those numbers also, but I still do not know what these numbers mean. I do not “feel” them. So I try to compare them to other causes of death such as people getting killed every month in (a) car accidents (9 per million) or (b) gun violence (8 per million) or (c) cancer (126 per million). These references help me to visualize the scale and impact of this pandemic.

So, while Covid-19 has killed about as many people in the US the last 4 weeks as people died in car accidents, in New York State the number of Covid-19 dead is about to exceed those who died of cancer in this same period. The hardest hit place in the US, however, is not New York City (160 dead per million), but New Orleans (295 dead per million). The County or Parish of New Orleans, Louisiana has about 400,000 people or a little less than New Castle County in Delaware where I live, but New Orleans has 115 dead compared to 5 in New Castle County (9 dead per million).

There are a few bright spots and I want to close on those. Los Angeles (7 dead per million) and California (5 dead per million) are doing remarkable well as does Germany (11 dead per million). Despite physical separations from others, I feel closer to friends, family, and neighbors both overseas and across the street. With more than 10 feet distance we have impromptu get-togethers between the door and the end of the driveway of 4 different households. I am happy to know that my neighbor Joyce from Kenya is safe back home living quarantined across the street with her African friends from Mali. She runs Water for Life which is a small non-profit that provides clean drinking water for rural communities in Kenya. It makes me happy to know her as a neighbor across the street.

And then there are the true warriors who fight this virus while endangering themselves to help others. Here is a nurse from Spain whose photo at work I took from her Twitter feed. We are all surrounded by wonderful and beautiful people.

Sea Ice from Satellite at 20-m Resolution

I am a self-taught amateur on remote sensing, but it tickled my pride when a friend at NASA asked me, if I could tell a friend of his at NOAA on how I got my hands on data to produce maps of radar backscatter to describe how the sea ice near Thule Air Base, Greenland changes in time and space.

Wolstenholme Fjord, Greenland Feb.-5, 2017 from Sentinel-1 radar. The data are at 20-m resolution

Wolstenholme Fjord, Greenland Feb.-5, 2017 from Sentinel-1 radar. The data are at 20-m resolution

In about 4 weeks from today I will be working along a line near the red dots A, B, and C which are tentative locations to place ocean sensors below the sea ice after drilling through it with ice fishing gear. The colored line is the bottom depth as it was measured by the USCG Healy in 2003 when I was in Thule for the first time. Faint bottom contours are shown in gray.

I discovered the 20-m Sentinel-1 SAR-C data only 3 weeks ago. They are accessible to me (after making an account) via

https://scihub.copernicus.eu/dhus/#/home

where I then search for a specific geographic area and time frame using the following “product”

Product Type: GRD
Sensor Mode: IW
Polarization: HH

Screenshot on how I search for the Sentinel-1 SAR-C DATA.

Screenshot on how I search for the Sentinel-1 SAR-C DATA.

The more technical detail can be found at

https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar

where one also finds wonderful instructional videos on how to work the software.

The data file(s) for a typical scene are usually ~800 MB, however, for processing I use the free SNAP software (provided by European Space Agency) via a sequence of steps that result in a geotiff file of about 7 MB.

Screenshot of SNAP software and processing with [1] input and [2] output of the Feb.-5, 2017 data from Wolstenholme Fjord.

Screenshot of SNAP software and processing with [1] input and [2] output of the Feb.-5, 2017 data from Wolstenholme Fjord.

This .tiff file I then read with Fortran codes to tailor my own (quantitative or analyses) purposes.

Start of Fortran code to covert the SNAP output geotiff file into an ascii file with latitude, longitude, and backscatter as columns. The code has 143 lines plus 80 lines of comment.

Start of Fortran code to covert the SNAP output geotiff file into an ascii file with latitude, longitude, and backscatter as columns. The code has 143lines plus 80 lines of comment.

The final mapping is done with GMT – General Mapping Tools which I use for almost all my scientific graphing, mapping, and publications.

Please note that I am neither a remote sensing nor a sea-ice expert, but consider myself an observational physical oceanographer who loves his Unix on a MacBook Pro.

Working the Night shift aboard CCGS Henry Larsen in the CTD van in Aug.-2012. [Photo Credit: Renske Gelderloos]

Working the Night shift aboard CCGS Henry Larsen in the CTD van in Aug.-2012. [Photo Credit: Renske Gelderloos]

If only my next problem, working in polar bear country with guns for protection, had as easy a solution.

Polar bear as seen in Kennedy Channel on Aug.-12, 2012. [Photo Credit: Kirk McNeil, Labrador from aboard the Canadian Coast Guard Ship Henry Larsen]

Polar bear as seen in Kennedy Channel on Aug.-12, 2012. [Photo Credit: Kirk McNeil, Labrador from aboard the Canadian Coast Guard Ship Henry Larsen]

Sea ice and 2016 Arctic field work

The sea ice in the Arctic Ocean is quickly disappearing from coastal areas as we are entering the summer melt season. This year I follow this seasonal event with nervous anticipation, because in October and November we will be out at sea working north of northern Alaska. We plan to deploy a large number of ocean sensors to investigate how sound propagates from the deep Arctic Ocean on to the shallow Chukchi Sea. This figure shows our study area with the ice cover as it was reported yesterday from space:

Ice concentration for June 14, 2016 from SSM/I imagery. Insert show study area to the north of Alaska and planned mooring locations (red box).

Ice concentration for June 14, 2016 from SSM/I imagery. Insert show study area to the north of Alaska and planned mooring locations (red box).

Zooming in a little further, I show the coast of Alaska along with 100 and 1000 meter contour of bottom depth over a color map of ice concentrations:

Ice concentrations from SSM/I to the north of norther Alaska with planned mooring locations across the sloping bottom. The 100 and 1000 meter contours are shown in gray with blue and red symbols representing locations of ocean and acoustic sensors, respectively.

Ice concentrations from SSM/I to the north of norther Alaska with planned mooring locations across the sloping bottom. The 100 and 1000 meter contours are shown in gray with blue and red symbols representing locations of ocean and acoustic sensors, respectively.

My responsibilities in this US Navy-funded project are the seven densely packed blue triangles. They indicate locations where I hope to measure continuously for a year ocean temperature, salinity, and pressure from which to construct sections of speed of sound and how it varies in time and space. I will also measure ice draft as well ice and ocean currents from which to estimate the roughness of the sea ice over time. Sea ice and ocean properties both impact sound propagation from deep to shallow water and vice versa.

A first question: What will the ice be like when we get there? This is the question that has the 40 or so people all working on this project anxiously preparing for the worst, but how can we expect what challenges are to come our way?

Doing my homework, I downloaded from the National Snow and Ice Data Center all gridded maps of ice concentrations that microwave satellites measured almost daily since 1978. Then I crunch the numbers on my laptop with a set of kitchen-sink Unix tools and code snippets such as

set ftp = 'ftp://sidads.colorado.edu'
set dir = 'pub/DATASETS/nsidc0081_nrt_nasateam_seaice/north'
...
wget -r -nd -l1 --no-check-certificate $ftp/$dir/$year/$file

along with fancy and free Fortran and General Mapping Tools to make the maps shown above. With these tools and data I can then calculate how much sea ice covers any area at any time. The result for custom-made mooring area at almost daily resolution gives a quick visual that I use to prepare for our fall 2016 expedition. The dotted lines in the top panel indicate the dates we are in the area.

Time series of daily ice concentration in the study area for different decades from January-1 through Dec.-31 for each year from 1980 through 2015. Panels are sorted by decade. The red curve is for 2015 and is shown for comparison in all panels.

Time series of daily ice concentration in the study area for different decades from January-1 through Dec.-31 for each year from 1980 through 2015. Panels are sorted by decade. The red curve is for 2015 and is shown for comparison in all panels.

The story here is well-known to anyone interested in Arctic sea ice and climate change, but here it applies to a tiny spec of ocean between the 100 and 1000 meter isobath where we plan to deployed ocean sensors for a year in the fall of 2016. For the two decades of the last century, the ice cover looks like a crap shoot with 80% ice cover possible any month of the year and ice-free conditions unlikely but possible here or there for a week or two at most. The situation changed dramatically since about 2000. During the last six years our study area has always been free of ice from late August to early October, however, our 2016 expedition is during the transition from ice-free October to generally ice-covered early November, but, I feel, our saving grace is that the sea ice will be thin and mobile. I thus feel that we probably can work comfortable on account of ice for the entire period, but the winds and waves will blow us away …

Weather will be most uncomfortable, because fall is the Pacific storm season. And with little or only thin ice, there will be lots and lots of waves with the ship pitching and rolling and seeking shelter that will challenge us from getting all the work done even with 7 days for bad weather built into our schedule.

I worked in this area on larger ships in 1993, 2003, and in 2004. Here is a photo that Chris Linder of Woods Hole Oceanographic Institution took during a massive storm in the general vicinity in October of 2004. The storm halted all outside work on the 420 feet long USCGC Healy heading into the waves for 42 long and miserable hours:

Icebreaker taking on waves on the stern during a fall storm in the Beaufort Sea in October 2004. [Photo Credit: Chris Linder, Woods Hole Oceanographic Institution]

Icebreaker taking on waves on the bow during a fall storm in the Beaufort Sea in October 2004. [Photo Credit: Chris Linder, Woods Hole Oceanographic Institution]

Oh, I now also recall that during this four-week expedition we never saw land or the sun. It was always a drizzly gray ocean on a gray horizon. The Arctic Ocean in the fall is an often cruel and inhospitable place with driving freezing rain and fog.

How to Power Modern Economies: Read Your Meter

Read you meter at home. This fun-filled advice was given by Sir David MacKay in a wonderful TEDx talk about how we heat our homes, get to work, run our computers, and how it all scales across countries and continents. The idea is really about how we run our lives while also trying to pass on a livable planet to our grand-children without the politically correct “greenwash” and self-righteous “claptraps”. Read your meter, do some algebra, and embrace the adventure to explore your home, your life, and the energy it all takes. If you read this far, watch the movie

David MacKay taught physics and information theory at the University of Cambridge in England. I learnt of him via Ruth Mottram in one of her many tweets. Dr. Mottram studies climate impacts of Greenland glaciers and works at the Danish Meteorological Institute. The tweet made me buy the book “Information Theory, Inference, and Learning Algorithm’s” that David MacKay wrote a few years back. It arrived today.

What piqued my interest was the advanced math that goes into designing networks that send and transform information such telephone calls via wireless, computer networks, and how to deal with imperfect channels of communication. My marriage comes to mind, too, because what I say is not always what I mean which is not always what my wife hears and vice versa, but I digress. Imperfect communication channels are one challenge we will face in an experiment to explore acoustic underwater data transmissions that hopefully will take place next year out of Thule Air Force Base, Greenland. Water and ice are imperfect communication channels that we need to use wisely to make our whispers carry far. Try to talk to a person across a busy street in Manhattan with all its hustle and bustle; you need to find something smarter and more effective than just simple shouting.

David MacKay wrote a second book that is close to his TEDx talk and is called “Sustainable Energy without the Hot Air.” Experimenting at home like any good physicist does, he discovers that “… the more often I read my meter, the less gas I use!”

There is so much more to this man, his work, and ideas as a physicist with a keen interest in the big picture without skipping the details. Sadly, he died yesterday of cancer too early only 48 years of age.

Thule, Greenland in Sharp Focus

I want to fly like an eagle
To the sea
Fly like an eagle
Let my spirit carry me

Steve Miller Band, 1976

The eagle “sees” the ground, because the twinkling sensation of light tickles her nerves. Today’s cameras work without the twinkle and tickle. They store numbers (digits) that approximate the amount of light passing through the lens. Satellite sensors work the same way. The data they beam to earth give me the soaring feeling of flying like an eagle, but there is more to the bits and bytes and digits sent home from space to our iPhones, laptops, and the internet.

Aerial photo taken Oct.-13, 1860 of Boston, MA by J.W. Black.

Aerial photo taken Oct.-13, 1860 Boston, MA from a balloon by J.W. Black.

The Metropolitan Museum of Art in New York houses the earliest existing aerial photo that was taken from a balloon hovering 600 meters above Boston, Massachusetts. Within a year the American Civil War broke out and this new technology became an experimental tool of war. It advanced rapidly, when air craft replaced the balloon during the First World War. Sharp photos of bombed-out battle and killing fields along the entire Western Front in France were taken by both Allied and German soldiers every day. Placing these photos on a map for efficient analyses of how a land- sea- or ice-scape changes over time, however, was impossible, because photos do not record precise locations.

Modern satellite photos are different. We now have fancy radar beams, computers, and several Global Position Systems (GPS) with atomic clocks to instantly calculation satellite tracks every second. This is why we now can both take photos from space AND map every dot or pixel that is sensed by the satellite moving overhead at 17,000 miles an hour snapping pictures from 430 miles above. The camera is so good that it resolves the ground at about 45 feet (15 meters). This is what such a (LandSat) picture looks like

LandSat photo/map of Thule, Greenland Mar.-17, 2016. The airfield of Thule Air Force Base is seen near the bottom on the right. The island in ice-covered Westenholme Fjord is Saunders Island (bottom left) while the glacier top right is Chamberlin Gletscher.

LandSat photo/map of Thule, Greenland Mar.-17, 2016. The airfield of Thule Air Force Base is seen near the bottom on the right. The island in ice-covered Westenholme Fjord is Saunders Island (bottom left) while the glacier top right is Chamberlin Gletscher.

Everyone can download these photos from the United States Geological Survey which maintains a wonderful photo and data collection archive at

http://earthexplorer.usgs.gov

but the tricky part is to turn these images or photos into maps which I have done here. More specifically, I wrote a set of c-shell and nawk scripts along with Fortran programs on my laptop to attach to each number for the light sensed by the satellite (the photo) another two numbers (the map). These are latitude and longitude that uniquely fix a location on the earth’s surface. A “normal” photo today has a few “Mega-Pixels,” that is, a few million dots. Each scene of LandSat, however, has about 324 million dots. This is why you can discern both the runways of Thule Air Force Base at 68 degrees 45′ West longitude and 76 degrees 32′ North latitude. The pier into the ice-covered ocean is just a tad to the south of Dundas Mountain at 68:54′ W and 76:34′ N. A scale of 5 kilometers is shown at the top on the right. For spatial context, here is a photo of the pier with the mountain in the background, that is, the object shown in the photo such as mountain, ship, and Helen serves a rough, but imprecise reference:

Dr. Helen Johnson in August 2009 on the pier of Thule AFB with CCGS Henry Larsen and Dundas Mountain in the background. [Credit: Andreas Muenchow]

Dr. Helen Johnson in August 2009 on the pier of Thule AFB with CCGS Henry Larsen and Dundas Mountain in the background. [Credit: Andreas Muenchow]

This photo shows the airfield and Saunders Island

Thule AFB with its airport, pier, and ice-covered ocean in the summer. The island is Saunders Island. The ship is most likely the CCGS Henry Larsen in 2007. [Credit: Unknown]

Thule AFB with its airport, pier, and ice-covered ocean in the summer. The island is Saunders Island. The ship is most likely the CCGS Henry Larsen in 2007. [Credit: Unknown]

The satellite image of the ice-covered fjord with Thule, Saunders Island, and Chamberlin Gletschers shows a richly texture field of sea ice. The sea ice is stuck to land and not moving except in the west (top left) where it starts to break up as seen by the dark gray piece that shows ‘black’ water peeking from below a very thin layer of new ice. There is also a polynya at 69:15′ W and 76:39′ N just to the south of an island off a cape. A polynya is open water that shows as black of very dark patches. A similar albeit weaker feature also shows to the east of Saunders Island, but it is frozen over, but the ice there is not as thick as it is over the rest of Westenholme Fjord. I suspect that larger tidal currents over shallow water mix ocean heat up to the surface to keep these waters covered by water or dangerously thin ice. There are also many icebergs grounded in the fjord. They cast shadows and from the length of these shadows one could estimate their height. Here is another such photo from 2 days ago:

LandSat photo/map of Thule, Greenland Mar.-21, 2016. The airfield of Thule Air Force Base is seen near the bottom on the right. The island in ice-covered Westenholme Fjord is Saunders Island (bottom left) while the glacier top right is Chamberlin Gletscher.

LandSat photo/map of Thule, Greenland Mar.-21, 2016. The airfield of Thule Air Force Base is seen near the bottom on the right. The island in ice-covered Westenholme Fjord is Saunders Island (bottom left) while the glacier top right is Chamberlin Gletscher.

I am using the satellite data and maps here to plan an experiment on the sea ice of Westenholme Fjord. Next year in March/April I will lead a team of oceanographers, engineers, and acousticians to place and test an underwater network to send data from the bottom of the ocean under the sea ice near Saunders Island to the pier at Thule and from there on to the internet. We plan to whisper from one underwater listening post to another to communicate over long ranges (20-50 kilometers) via a network of relay stations each operating smartly at very low energy levels. We will deploy these stations through holes drilled through the landfast ice 1-2 meters thick. The work is very exploratory and is funded by the National Science Foundation. Wish us luck, as we can and will use it … along with aerial photography that we turn into maps.