AnDevCon 2017 – How Yahoo Finance Scales on Android – Vikram Bodicherla

I saw Vikram‘s talk at Droidcon NYC 2016 talk about app performance, and wanted to make sure I made it to his talk.  He talks about overall processes for a successful development team along with tools and tips for shipping apps.

Developers shouldn’t be afraid to dip your toes into “product”.
Contribute to feature definition to make your voice heard as a user and for technical implementation.

Understanding your users – Fantasy Football Example:
You should be aware of who is using your app. You might think everyone using a fantasy football app are very serious users, but we find that 12% randomly pick, and 60% are first year users. Try to find patterns between different types of users of your app. Build personas of your users. Building personas of your users is a way to explain behavior that would be hard to explain otherwise. Give these personas name and put them up on the wall in your office. Know who your users are and design for them.

Personas for fantasy football includes:

  • Seasoned pros
  • People that are forced to join by their friends
  • People that are serious, but learning
  • “average” users

Data Driven Decisions via Experiments:
DAUs and Downloads are really “Vanity Metrics”, since they don’t gain a ton of insight. Data driven insights are real; not opinions. Wrong decisions can be validated quickly, so feel empowered to test, measure, and grow. Instead of doing costly user research, you can push something to a small group of users, and gain data there vs. expensive user research.

Each experiment should have:

  • A hypothesis
  • An owner
  • A deadline for the end of the experiment.
  • Planned removal of the code from your app. You learned something from it, now “Let it go!!!”

Scrum
Planning is a good investment. You can see how long things will take, determine dependencies, and people can show interest in pieces of the work.  Agile is important because it’s not adverse to change like Waterfall is.

Types of stories in Scrum:

  • User story
  • Engineering story

Scrum story checklist:

  • Clear definition
  • Acceptance criteria
  • Edge cases and error scenarios
  • Analytics
  • Animations and design

Grooming

  • should be (1x/week for 1 hour)
  • It helps the team to understand stories -> Determine level of effort for stories (Points)
  • Stories should have reasonable detail. Be very critical and make sure the story is well defined.
  • Don’t get hung up on one story. If your getting hung up, move on and take that offline so you don’t waste your entire grooming on one thing.
  • NEVER write stories during Grooming.
  • NEVER point a story that doesn’t have all the details.

If you get roll-over and carry-over after a sprint consistently, you’re doing something wrong and should revisit your processes.

Daily stand-ups should be no greater than 10 minutes and should provide high level updates. Low level updates should be taken “offline”, or “parking lotted”.

At the end of the sprint, you can hold a retrospective and demo completed stories.  Please prepare up front for this so we don’t have to wait for you to launch the emulator or recompile your project.

“If you’re are not practicing scrum in sprit, you are better off not doing it.”

Continuous Delivery

  • Minimizing the time between “code complete” and “deployable”
  • deployable != deploy
  • This can only be achieved with continuous integration (CI)
  • If you do deploy immediately to your users, you get immediate feedback, but not always the best for your users. Vikram recommend every 2 weeks on average, but depends.
  • Have internal alpha and beta programs for internal users to try the app before it goes to production.

Pull Request (PR) Tools

  • facebook/mention-bot – Can tag reviewers based on “git blame” history
  • SonarCube
  • https://codecov.io for Code Coverage
  • Danger -> Keep PRs <500 lines, PR message should be >25 chars.
  • Emulator in an iframe, running the code in the PR.  -> https://appetize.io Removes need to compile and install.

APK storage and deployment:

Culture

  • How you act/think/interact. Keep high standards, because you are what your lowest standard is.
  • Ownership is important. Communicate deviations and compromises so it’s not lost. Proactively check app quality issues and triage as soon as you can.
  • Set goals, but aim for 70%. If you get 100%, then your goals are not high enough.

Meeting etiquite

  • Everyone needs to have value, or they shouldn’t be there
  • Reject meetings if they are not needed
  • Meeting owner should have adgenda, goals and come out with action items.
  • There should be a hard start and end type.

Code is a liability. Once it’s out there, you have to support it. Changing code is expensive.  Deleting code is a good thing.

Feature Flags – Client-side feature flags are important, along with remote feature flags.  Client-side flags should take priority, but if it’s on, then use the remote flag.

Running Multiple Android Emulators Simultaneously on OSX

I had a long standing issue where I could only launch two emulators on my Macbook Pro simultaneously.  Any other emulators would launch and get a black screen and wouldn’t load.  I accepted this as fact, and carried on with life.

screen-shot-2016-09-07-at-7-11-45-pm

Finally today I found the need to figure out “why” this was the case as I wanted to start sharding my Android UI tests to speed them up.  I was given the suggestion that it could be the # of CPUs or the amount of RAM.  If you look in the logs you may see something like this (if you launch the emulator from android avd):

screen-shot-2016-09-07-at-7-15-46-pm

or you may see:

“HAXM does not have enough memory remaining to load this AVD.”

In order to change your memory allocation for HAXM, you need to re-install it, so run the installer again which is a DMG at: [$ANDROID_HOME/sdk/extras/intel/Hardware_Accelerated_Execution_Manager/ ] and this time select more memory. [HAXM Technical Docs]screen-shot-2016-09-07-at-8-27-05-pm

But… before you install, make sure you have the latest version. The latest version as of September 7th, 2016 is 6.0.3 (Which was released on June 21, 2016).
screen-shot-2016-09-07-at-7-33-01-pm

On my Macbook, I was given 2048MB of memory by default which allowed me to run 2 emulators simultaneously (as seen above).screen-shot-2016-09-07-at-6-58-46-pm

I have 16GB total memory though, so lets make it so I can run a bunch of emulators at once by allocating 8GB 🙂
8192
confirm

Here is my Macbook Pro running 6 Android emulators simultaneously after the changes:
screen-shot-2016-09-07-at-8-17-17-pm

Success!!!  May the sharding begin!

Related Links:

Using the Android Emulator for Continuous Integration

Screen Shot 2016-04-23 at 9.06.54 PM

I spent the last few weeks figuring out how to get our Android Tests running on our Jenkins CI (Continuous Integration) server using the latest Android Emulator. I am sharing what I learned so that I can hopefully help someone else and so I can refer back to it in the future.

Result of this tutorial/example:

You will be able to programmatically create an Android Emulator AVD (Android Virtual Device), start it, wait for it to load, run your tests, then kill the emulator, generate a Spoon html test report.

Screen Shot 2016-04-23 at 8.55.17 PM

Assumptions for this example:

  • You have the Android SDK configured on this machine with:
    • SDK components for API 23
    • The latest build tools
    • Google apis support
    • API 23, x86_64 with Google APIs Image
    • HAXM support installed
  • You have an Android build flavor named “qa”.
  • You are running your tests on a “debug” build.
  • You have built the APK and test APK with:
    • ./gradlew clean app:assembleQaDebug and app:assembleQaDebugAndroidTest
  • You have Android tests in a Java package called “integration_tests”
  • spoon-runner-1.2.0-jar-with-dependencies.jar exists in your working directory.
  • You want to use an API 23, x86_64, Google APIs enabled AVD (Note, you could easily change these, but this is what this example uses)

To download all the necessary SDK components referenced above, you can run the following command, or just download the components using the UI version of the Android SDK Manager.  NOTE: These commands aren’t scripted for a CI server.  You have to manually accept licensing terms:

android update sdk --no-ui --all --filter 'platform-tools,extra-android-support,build-tools-21.1.1,extra-android-m2repository,extra-google-m2repository,android-23,addon-google_apis-google-23,sys-img-x86_64-google_apis-23'

This Android test execution script will:

  • Forcefully create an AVD for this test (so that it is a fresh device every time)
  • Use port 6000 for the emulator (This is arbitrary, but by default, it is 5554 and assigned by Android).
  • Start the emulator for the AVD on a specific port (so we can target the device by serial with “emulator-{PORT}”
  • Ensure the emulator started by polling for the emulator to be booted
  • Use Spoon to run the tests in a specific package
  • Kill this emulator instance after our tests complete
#CONFIG
FLAVOR="qa"
PACKAGE_NAME="com.example.myapp"
AVD_NAME="integration-tests"
PORT=6000

##############################################################################

#Ensure the Test APK is built already.
TEST_APK_FILE="app/build/outputs/apk/app-${FLAVOR}-debug-androidTest-unaligned.apk"
if [ ! -f "${TEST_APK_FILE}" ]
then
	echo "androidTest APK doesn't exist, exiting.  Make sure you run ./gradlew app:assembleQaDebug app:assembleQaDebugAndroidTest"
exit
else
	echo "androidTest APK Exists, continuing"
fi

#Calculate the Serial Number of the emulator instance
SERIAL=emulator-${PORT}

echo "Creating (forceful) AVD with name ${AVD_NAME}"
# We have to echo "no" because it will ask us if we want to use a custom hardware profile, and we don't.
echo "no" | android create avd \
    --name "${AVD_NAME}" \
    --target "android-23" \
    --abi "google_apis/x86_64" \
    --skin "WXGA720" \
	--force
echo "AVD ${AVD_NAME} created."

#Start the Android Emulator
#"2>&1" combines stderr and stdout into the stdout stream
START_EMULATOR="emulator \
	-avd ${AVD_NAME} \
	-netspeed full \
	-gpu on \
	-netdelay none \
	-port ${PORT}"

echo $START_EMULATOR
$START_EMULATOR 2>&1 &

#Ensure Android Emulator has booted successfully before continuing
EMU_BOOTED='unknown'
while [[ ${EMU_BOOTED} != *"stopped"* ]]; do
    sleep 5
    EMU_BOOTED=`adb -s ${SERIAL} shell getprop init.svc.bootanim || echo unknown`
done

duration=$(( SECONDS - start ))
echo "Android Emulator started after $duration seconds."

# Get the file name of the QA APK (we need to cope with the version number changing)
APK_LIST_AS_TEXT="$(ls -t app/build/outputs/apk/*-android-debug-*-${FLAVOR}.apk)"
APK_FILES=( $APK_LIST_AS_TEXT )

# Use the Spoon utility as a test runner
SPOON_COMMAND="java -jar spoon-runner-1.2.0-jar-with-dependencies.jar \
	-serial ${SERIAL} \
	--apk ${APK_FILES[0]} \
	--test-apk ${TEST_APK_FILE} \
	--no-animations \
	--fail-on-failure \
	--debug
echo "Running: ${SPOON_COMMAND}"
${SPOON_COMMAND}

#Stop the Android Emulator
echo "Killing the Android Emulator with serial: ${SERIAL}"
adb -s ${SERIAL} emu kill

Final notes & tips:

  • Because the Android Emulator’s serial number is in the format of “emulator-{SERIAL}”, it is very important to start the emulator on a specific port (using the “-port {PORT}” param) so that you can reference the image by serial number.  This allows you to launch more than one emulator on the same build machine.
  • Command to list available Android platforms
    android list targets
  • Command to list available AVDs
    android list avd
  • Command to list all available SDK components that you can –filter for in the “update sdk” command:
    android list sdk --all --extended

If you have any questions or comments, let me know on Twitter at @HandstandSam.

Continuous Integration at LesFurets.com – Richmond Java Users Group (RJUG)

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” – 1st principal of the Agile Manifesto

This presentation by Raphaël Brugier (@rbrugier) at the Richmond Java Users Group last night (Feb 17th) was really good and highlighted:

  • Continuous Integration.  A case study of LesFurets.com. 40,000 unit tests running in 3 minutes.  Selenium tests running in 15 minutes.  Passing tests == shippable code.
  • They went from shipping a feature in a month, to as soon as ready (possibly only a few days).
  • A different kind of Git Flow using git octopus. – https://github.com/lesfurets/git-octopus
  • They have new developers hit the deploy button their first day.  Share deployment responsibilities within the dev team.
  • “If it hurts, do it more often.” – In regards to doing a deployment/release.
  • Work on feature branches.  Ship features as ready, not based on sprints. (git octopus facilitates this)
  • Detect feature branch merge conflicts after every push and make sure it’s resolved ASAP.
  • QA is owned by developers and sign off is given by product when dev demos it.
  • and more…

I would highly recommend watching the recording of the event.  Heads up: The speaker had a French accent and the video is from my MacBook’s camera, however, you get to watch it and otherwise you wouldn’t be able to :-p.

YouTube Link: https://www.youtube.com/watch?v=uTOPvC3lV1Q

Slides: http://www.slideshare.net/raphaelbrugier/continuous-delivery-journey-at-lesfuretscom

Meetup Info: http://www.meetup.com/Richmond-Java-Users-Group/events/226909145/