we will now get into collections – this is what lends itself to why its a great language for Data Engineering
tuple is an immutable ordered containers of values of different types
scala> val mytuple = ("hi" ,"this" , "is" ,"a" , "tuple" ,5 , 6 )
mytuple: (String, String, String, String, String, Int, Int) = (hi,this,is,a,tuple,5,6)
scala> val maptypetuple = 1 -> "hi"
maptypetuple: (Int, String) = (1,hi)
val anothertuple = ("hi" ,1 , 2.0)
anothertuple: (String, Int, Double) = (hi,1,2.0)
scala> anothertuple._1
res10: String = hi
scala> anothertuple._2
res11: Int = 1
scala> anothertuple._3
res12: Double = 2.0
scala> anothertuple._4
^
error: value _4 is not a member of (String, Int, Double)
// accessing tuple values - note it starts with 1 as opposed to 0 like list or arrays
// another way
scala> val (myfristvar, secondvar , 3rdvar) = anothertuple
^
error: Invalid literal number
scala> val (my1var, secondvar , my3rdvar) = anothertuple
my1var: String = hi
secondvar: Int = 1
my3rdvar: Double = 2.0
scala>
// use placeholder "_" to fill in a position where you dont want to pick the element
scala> val (my1var, _ , my3rdvar) = anothertuple
my1var: String = hi
my3rdvar: Double = 2.0
scala>
// to get size of tuple
scala> anothertuple.productArity
res19: Int = 3
// to iterate
scala> anothertuple.productIterator.foreach{i => println(i)}
hi
1
2.0
// you can use the tupled over to the method converted to function ...see below
scala> (math _).tupled(10.0,20.0,(a,b) => a -b )
res30: Double = -10.0
List – in scala is an immuatble single linked list with the last value indicated by Nil
the construction operator is ::, construting list this way requires you to use Nil at the end
scala> val mylist = 1::2::3::4::5
^
error: value :: is not a member of Int
scala> val mylist = 1::2::3::4::5 ::Nil
mylist: List[Int] = List(1, 2, 3, 4, 5)
// or the other way is by invoking list constructor
scala> val mynewlist = List(1,2,3,4,5)
mynewlist: List[Int] = List(1, 2, 3, 4, 5)
scala> val mywordlist = List("one","two","three","four","five")
mywordlist: List[String] = List(one, two, three, four, five)
scala> mynewlist ++ mywordlist
res31: List[Any] = List(1, 2, 3, 4, 5, one, two, three, four, five)
// the ++ operator works on all iterables
scala> mynewlist ::: mywordlist
res32: List[Any] = List(1, 2, 3, 4, 5, one, two, three, four, five)
// the ::: operator is specific to list - both are used for concat - note it //automatically converts to type Any , because it encounters both - type inference at // work here
scala> mynewlist zip mywordlist
res33: List[(Int, String)] = List((1,one), (2,two), (3,three), (4,four), (5,five))
// zip combines the two list into a list of tuples
scala> List(mynewlist,my2ndlist).flatten
res34: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
// can use the flatten method to combine lists
/other list operations
scala> mynewlist.max
res35: Int = 5
scala> mynewlist.min
res36: Int = 1
scala> mynewlist.product
res37: Int = 120
scala> mynewlist.sum
res38: Int = 15
scala> mynewlist.take (2)
res39: List[Int] = List(1, 2)
scala> mynewlist.drop(1)
res40: List[Int] = List(2, 3, 4, 5)
scala> mynewlist.size
res41: Int = 5
scala> mynewlist.head
res42: Int = 1
scala> mynewlist.tail
res43: List[Int] = List(2, 3, 4, 5)
scala> mynewlist.contains(6)
res44: Boolean = false
scala> mynewlist contains 6
res45: Boolean = false
scala> mynewlist contains 5
res46: Boolean = true
scala> mynewlist.reverse
res47: List[Int] = List(5, 4, 3, 2, 1)
notice the head returns the first element . but the tail returns everything other than head
scala> mynewlist.splitAt(2)
res49: (List[Int], List[Int]) = (List(1, 2),List(3, 4, 5))
higher order functions with List
foreach is a procedure – so it does not return a value
scala> val x = mynewlist.foreach(_ + 2)
x: Unit = ()
// not x doesnot have anything becuase foreach does not return anything
scala> mynewlist.foreach(println (_))
1
2
3
4
5
scala> val list = List(1,2,3,4,5)
list: List[Int] = List(1, 2, 3, 4, 5)
scala> list.scanRight(0)(_ - _)
res3: List[Int] = List(3, -2, 4, -1, 5, 0)
scala> list.scanRight(2)(_ - _)
res4: List[Int] = List(1, 0, 2, 1, 3, 2)
scala> list.scanRight(20)(_ - _)
res5: List[Int] = List(-17, 18, -16, 19, -15, 20)
scala> list.scanLeft(0)(_ - _)
res6: List[Int] = List(0, -1, -3, -6, -10, -15)
scala> list.scan(0)(_ - _)
res7: List[Int] = List(0, -1, -3, -6, -10, -15)
scala> list.foldRight(0)(_ - _)
res8: Int = 3
scala> list.foldLeft(0)(_ - _)
res9: Int = -15
scala> list.foldRight(0)(_ - _)
res10: Int = 3
scala> list.fold(0)(_ - _)
res11: Int = -15
scala> list.reduceLeft(_-_)
res12: Int = -13
scala> list.reduceRight(_-_)
res13: Int = 3
scala> list
res14: List[Int] = List(1, 2, 3, 4, 5)
the above code shows the operation for scanleft scanright ,scan ,foldleft, foldright, fold, reduceRight, reduceLeft . Notice that other than scan all operations return the last value .
mylist
res27: List[String] = List(Hello, World, From, Sunil)
scala> mylist.startsWith("Hello")
res30: Boolean = false
val mycomplist = List("Hello")
mycomplist: List[String] = List(Hello)
scala> mylist startsWith mycomplist
res33: Boolean = true
scala> mylist forall (_ != "today")
res34: Boolean = true
As you can see above the you can use the startsWith to compare string , note you cannot pass the actual value of the first element , it has to be another list you are comparing with. Endswith also works in a similar way.
Next we will look at Map …Map is essentially a collection of key value pairs , which are in itself tuples , you can create these with the -> operator or with parantheses as shown below . Note , map which applies a function to a given collection is different from the data structure Map …dont get confused between the two.
scala> val mymap = map(1 -> "one" , 2 ->"two", (3,"three"))
^
error: not found: value map
scala> val mymap = Map(1 -> "one" , 2 ->"two", (3,"three"))
mymap: scala.collection.immutable.Map[Int,String] = Map(1 -> one, 2 -> two, 3 -> three)
// you can look up values by specifying the key , also use the contains method // to check for the existence of a key
scala> mymap(2)
res37: String = two
scala> mymap(4)
java.util.NoSuchElementException: key not found: 4
at scala.collection.immutable.Map$Map3.apply(Map.scala:335)
... 28 elided
scala> mymap.contains(3)
res39: Boolean = true
scala> mymap.contains(4)
res40: Boolean = false
Higher order functions like foreach, map, reduce will work with Maps just like lists but we must bear in mind that Map is made up of two element tuples so the function has to work on tuple
you can convert Lists to Map , using a combination of zip and toMap method, see below for this in action
//we already have a map defined , lets see how we can build a similar map with
//list
scala> mymap
res41: scala.collection.immutable.Map[Int,String] = Map(1 -> one, 2 -> two, 3 -> three)
scala>
scala> val nolist = List(1,2,3)
nolist: List[Int] = List(1, 2, 3)
scala> val wordlist = List("one","two","three")
wordlist: List[String] = List(one, two, three)
scala> val mynewmap = (nolist zip wordlist).toMap
mynewmap: scala.collection.immutable.Map[Int,String] = Map(1 -> one, 2 -> two, 3 -> three)
// use == op for comparing maps
scala> mymap == mynewmap
res42: Boolean = true
val nlist = mynewmap.keySet.toList
nlist: List[Int] = List(1, 2, 3)
scala> val wlist = mynewmap.values.toList
wlist: List[String] = List(one, two, three)
// use keySet and Values to extract keys and values into lists
Sets are similar to lists except they are unordered and the elements are unique – see below , it automatically removes the
scala> val myset = Set(1,2,3)
myset: scala.collection.immutable.Set[Int] = Set(1, 2, 3)
scala> val myset = Set(1,2,3,3)
myset: scala.collection.immutable.Set[Int] = Set(1, 2, 3)
scala> val myset = Set(3,2,1,2)
myset: scala.collection.immutable.Set[Int] = Set(3, 2, 1)
scala>
// notice all of these invoke immuatble collection - so the set , map and list
// are immutable . To create the mutable ones - invoke with the right name // // space as shown below
val mymutablemap = collection.mutable.Map(1->"one",2->"two",3->"three")
mymutablemap: scala.collection.mutable.Map[Int,String] = HashMap(1 -> one, 2 -> two, 3 -> three)
scala> val mymutableset = collection.mutable.Set(1,2,3)
mymutableset: scala.collection.mutable.Set[Int] = HashSet(1, 2, 3)
val mymutablelist = collection.mutable.Buffer(1,2,3)
mymutablelist: scala.collection.mutable.Buffer[Int] = ArrayBuffer(1, 2, 3)
// note mutableList is created using the Buffer method
//converting to immutable list is easy. - you just call the toList or toMap or //toSet method on the mutable object
scala> val myimmutablelist = mymutablelist.toList
myimmutablelist: List[Int] = List(1, 2, 3)
// going back to mutable list is a bit involved
scala> val listbuilder = List.newBuilder[Int]
listbuilder: scala.collection.mutable.Builder[Int,List[Int]] = ListBuffer()
scala> myimmutablelist.foreach(listbuilder+=_)
scala> list
list listbuilder
scala> listbuilder.result
res48: List[Int] = List(1, 2, 3)
Arrays are fixed lengths but the contents can be modified
option is a collection used to capture presence or absence of a value , none is a special value of option …with option you need to write how to handle the logic – use the getOrElse method
use util.Try for try catch style program , Success and Failure and keywords and you can write a match statement and check with a case statement to handle logic in both cases
in the next page we will get into classes