2014
National Online Dialogue on
Immigration Reform
with the technology community
conducted by
Reframe It
in collaboration with
The Knight Foundation
TechCrunch
Silicon Valley Community Foundation
The Miami Foundation
Executive Summary
What happens when thousands of technology community members participate in a collective intelligence process to provide their insights into immigration policy? Can a controlled experiment show what would happen if technology community members spent a full day deliberating about immigration reform on the basis of balanced information? How would their attitudes about immigrants and immigration policy change after a day-long of informed dialogue about policy alternatives?
Seven experts on immigration policy were consulted over a nine-month period to develop briefing materials. More than two thousand technology community members examined the materials online and posted over 400 comments at specific points in the text. This line-by-line feedback led to more than 70 revisions. At that point, a sample of the tech community members was recruited participate in the controlled experiment. Incentives included the chance to win one of 30 tickets to a major TechCrunch Disrupt conference in San Francisco, London or New York, in addition to a small cash honorarium. Individuals in the panel were randomly assigned to be in either (a) the treatment group that received the vetted briefing materials and had a day of moderated online small-group discussion, or (b) the control group. There were 87 were people in the treatment group and194 in the control group.
Control groups were given a traditional survey twice over the period of a few months. Treatment groups were given these same surveys at the beginning. After taking the surveys, the treatment group also received 44 pages of briefing materials two days before the deliberations. They dialogued via moderated group video chats with fellow members of the tech community in small groups and have their group’s questions answered by experts (along with the questions from other groups) at a live forum. The experts include prominent authorities on immigration law and immigration public policy, listed below.
The results of the process show that tech community members became more compassionate and more sophisticated on the impact of immigrants on the American culture in the course of one day of informed dialogue about immigration policies and immigration reform. There was a large jump (18%) in support for finding a path to citizenship for DREAMers, and there was a sizeable jump (16%) in those who believed that having close family in the US should be important in issuing working visas. In addition, there was a 17% decrease in those who believed that immigrants threaten the nation’s culture (as well as a 6% increase in those who believed immigrants had a lot to offer to the cultural life of the US): There was a jump from 60% to 78% in finding a path to citizenship for DREAMers. Support for “Having close family in the US should be important in issuing work visas” moved from 51.8% to 67.9%.On a scale from 0 to 10, where 0 means that immigrants have a lot to offer to the US’s cultural life, 10 means that immigrants threaten the US’s culture, and 5 is exactly in the middle? 0-4 changed from 61.9 to 67.9; 5 changed from 10.7 to 11.9; 6-10 changed from 37.4 to 20.2.
After deliberation, participants were significantly more in favor of allowing a larger number of people to immigrate–but with more safeguards to ensure that the immigration system was not abused by any given company overusing the H1B process, and that H1B workers should have pay requirements increased to the median wage in a region by occupation. Technology community members after deliberation came out in favor of increasing access to citizenship for parents of US citizens and job skills get prioritized in future immigration policies.
(1) Background
Debates about immigration policies today are often rancorous, simplistic and ill informed. While immigration policies impact the tech community profoundly, often with consequences that were unintended by the policy’s framers, misinformation and disinformation have undeserved influence. Because of this, preventable and problematic unintended consequences go unchecked.
Our goal in this deliberative process was to bring a currently underutilized resource to these debates: the intelligence, creativity and problem-solving mindset of the tech community. We wanted both to harness fresh ideas about assumptions, arguments and possible reforms, and to test whether the tech community would support these reforms if they understood them. Our hope is that this will be the first of several opportunities in which people can be involved in a creative process of public policy ideation and debugging—a process which is structured in such a way as to be invulnerable to manipulation, perversion or capture by special interests.
Ordinary public opinion polls measure what the public thinks at the moment a poll is taken; in such surveys, many people haven’t had a chance to learn about and think about the issues before they’re surveyed, and as such, give off-the-cuff opinions. This can be problematic. George Bishop famously showed this when he surveyed Americans about the Public Affairs Act of 1975 and showed that 44% of Americans were willing to express a firm opinion for or against the act— even though the act didn’t exist and was made up for the purpose of the survey. Our approach is to measure what the people would think if they were thinking.
In our controlled experiment, members of the treatment group received a treatment modeled on a Deliberative Poll: they were was exposed to briefing materials that were vetted by experts with diverse perspectives and then participated in a series of moderated online small-group discussions with fellow members of the tech community. Afterwards they took a final survey to measure what the treatment group thought after going through this process. The results reflect statistically what the tech community would think about immigration policy after carefully weighing the alternatives. Our control group population was simply surveyed twice by conventional methods (at the same two points in time as the treatment group).
Reframe It collaborated with TechCrunch, Silicon Valley Community Foundation, Knight Foundation and Miami Community Foundation to learn what America’s technology community would think about immigration reform and immigration public policy choices if they took the time to think about the issues and talk them through for a day. Additionally, Venture Beat supported the initiative by inviting their readers to comment on the briefing materials, and Mind Media Productions supported our film editing. YouGov Polimetrics supported our sample recruitment.
After Knight Foundation provided a grant to Reframe It, a briefing committee was created. The members of the briefing committee represent multiple points of view on immigration policy in the US and on immigration reform. Members of the briefing committee were:
- Michael Clemens, Senior Fellow at the Center for Global Development
- Ron Hira, Associate Professor in the Department of Science, Technology, and Society/Public Policy at Rochester Institute of Technology
- Cheryl Little, Executive Director and Co-Founder of Americans for Immigrant Justice
- Norm Matloff, Professor of Computer Science at UC Davis
- Craig Montuori, Founder and Executive Director of PolitiHacks
- Hal Salzman, Professor of Planning and Public Policy at Rutgers University
- Manuel Santamaria, Director of Grantmaking at the Silicon Valley Community Foundation
This committee, in collaboration with Reframe It, drafted briefing materials. Once the materials were ready, Reframe It issued an open call for members of the tech community to add their creative ideas on immigration policy reform and the challenges it seeks to resolve by contributing annotations to the briefing materials using Reframe It’s software. TechCrunch announced the call for comments, along with Knight Foundation and Silicon Valley Community Foundation. Members of the U.S. tech community were invited to add their specific ideas line-by-line to the briefing materials. Tech Community members were invited to make new arguments, identify assumptions that that haven’t been identified or explored, and devise approaches that may not have occurred to policymakers and policy think tanks. They were asked to do something at which they have learned to excel in other contexts: find preventable bugs in the system, which could lead too serious problems down the line. Paraphrasing “Linus’s Law” in “The Cathedral and the Bazaar,” Eric Steven Raymond observed, “Given enough eyeballs, all bugs are shallow.” The goal of this venture was to leverage the tech community’s can-do spirit by bringing its creative energy to bear on the debugging of public policy related to immigration reform.
You can see the briefing document and read the more than 400 line-by-line annotations the tech community made on it by going here.
Participants were recruited via multiple means, including via TechCrunch, the polling firm YouGov/Polimetrix, and via twitter ads targeting technology community members. (A detailed description of this process is in the following section.) Technology community members who agreed to participate in the controlled experiment were randomly assigned to either the treatment group or the control group. Participants in the control group received an honorarium of $15, while participants in the treatment group received an honorarium of $50-$100. The control group took the survey at the beginning of the process and again at the end.
Members of the treatment group were given access to 44 page briefing materials created in collaboration with experts who disagreed on multiple aspects of immigration policy. The treatment group engaged in online small-group discussions with trained moderators. These concluded with an opportunity for each small group to ask a couple of questions to a group of experts during a plenary session who represented diverse points of view.
These experts participating in the plenaries were:
Jim Alexander. Jim Alexander is a Managing Shareholder at the immigration law firm Maggio and Kattar. As an expert in issues of corporate compliance in immigration law, e-Verify, and naturalization matters, he works with clients ranging from multinational corporations to start-up enterprises. He advises institutional and individual clients in the finance, defense, aviation, and technology fields on employment- and family-based immigration, removal defense, and asylum.
Ron Hira. Ron Hira is an assistant professor of public policy at Rochester Institute of Technology and a research associate at the Economic Policy Institute. He is the author of the book Outsourcing America, and his research interests include skilled labor markets, immigration, offshoring, and globalization. As a licensed engineer, he also serves as vice president for career activities for IEEE-USA.
Lisa Johnson-Firth. Lisa Johnson-Firth is the principal of Immigration First, an immigration and human rights law firm. She specializes in removal defense, asylum, U and T visas, naturalization, Convention Against Torture claims, and Violence Against Women Act petitions. Ms. Johnson-Firth also serves an adjunct professor of law at Georgetown University Law Center, where she focuses on Immigrant Women and Human Rights.
Sweta Khandelwal. Sweta Khandelwal is the principal attorney at the Law Office of Sweta Khandelwal, a firm specializing in US immigration and nationality law. She has been recognized as one of the “Top 40 Under 40” immigration lawyers in California by the American Society of Legal Advocates and is Chair of the Silicon Valley Chapter of the American Immigration Lawyers Association. In addition to Ms. Khandelwal’s professional experience in the field of immigration law, her personal experiences as an immigrant to the US have provided her with extensive knowledge of the challenges faced by employment-based immigrants.
David Leopold. David Leopold is founder and principal of David Wolfe Leopold and Associates, where he specializes in matters of immigration, visas, and citizenship. His work focuses on representation of health care institutions and physicians, deportation defense, and federal immigration litigation. He is an advocate for immigration reform and has served as the president of the American Immigration Lawyers’ Association.
Norm Matloff. Norm Matloff is a professor of computer science at UC Davis. He specializes in parallel processing, computer communication networks, data security, and statistics. He formerly worked as a database software developer in Silicon Valley and has written extensively on tech industry labor practices, focusing on the H-1B work visa. Dr. Matloff has also provided expert testimony on age and racial discrimination in the software industry to the US Senate and House of Representatives.
John Miano. John Miano has volunteered with the Center For Immigration Studies since 2008. His area of expertise is in guest worker programs, particularly in how they affect the technology work force. John Miano has testified on immigration issues before the Immigration and Claims Sub-Committee of the House Judiciary Committee as an expert on immigration. Mr. Miano has a BA in Mathematics from The College of Wooster and a JD from Seton Hall University. Mr. Miano is also the founder of the Programmers’ Guild, an organization committed to advancing the interests of technical and professional workers.
Pete Peterson. Pete Peterson is Executive Director of the Davenport Institute at Pepperdine University, where he specializes in public engagement. He previously served as the first executive director of Common Sense California, which promotes citizen engagement as a means for producing innovative policy decisions and better citizens. Building on his interest in citizen/government relationships, he has consulted on participatory planning and budgeting projects in cities across California. Pete Peterson was the 2014 Republican candidate for Secretary of State of CA.
Atul Singh. Atul Singh is a lawyer, current affairs lecturer, and founder and editor-in-chief of Fair Observer, a global affairs analysis site. He previously worked in corporate law and has spoken on a wide variety of topics in foreign policy, international affairs, and global economy. He is also a contributor to the Huffington Post and a columnist for Al Jazeera.
Andrew Wainer. Andrew Wainer is a Senior Immigration Policy Analyst for Bread for the World Institute. He has served as a researcher and analyst in immigrant communities and has contributed to Congressional Research Service reports, the Wall Street Journal, the Los Angeles Times, the National Journal, and the Detroit News, as well as peer-reviewed academic journals such as Migration and development in
(2) Experimental Process
Participant Recruitment
Participants in the experiment were recruited in ways designed to yield a pool that mirrored as closely as possible the demographics of the US technology community. These methods included:
- A newsletter sent by TechCrunch
- Two additional newsletters sent by TechCrunch with Twitter and Facebook replications
- Paid advertisement on Facebook and Twitter
- Recruitment by YouGov/Polimetrix
These four methods represented the bulk of all participation. Supplemental recruitment was conducted by making posts on online forums popular in the tech community:
- TechCrunch
- High Tech Rochester (linkedin)
- SF New Tech (linkedin)
- Tech Junction (linkedin)
- Smart Choices (yahoo group)
- Startup Hackathon (meetup group)
- ArtsTech SF (meetup group)
- SF New Tech (meetup group)
- San Francisco Big Data Science (meetup group)
- Reddit Technology
- Peoplesoft fans (yahoo group)
- NY Tech Meetup (meetup group)
- Tech in Motion New York (meetup group)
- NY EdTech (meetup group)
- Tech Plus
- JobstheWord – Jobs, Career News, Social Media News, Tech News
- High-Tech Product Strategy
- The Virtualization and Cloud Computing Group
- IT Tech Writers
- On Startups – The Community for Entrepreneurs
- Tech-Writer
- Hi Tech Happy Hour
- Tech Cocktail Group
All participants recruited through all means were randomly assigned to the treatment and control groups.
In order to secure an appropriate sample of tech community members, participants were offered cash incentives to participate.[1] Participants were offered $50-$100 if they completed deliberation as part of the treatment group.[2] Participants in the control group were offered $15. Participants in both the treatment and control groups were eligible to win one of 30 grand prizes via a raffle conducted by the Silicon Valley Community Foundation for the project. The grand prizes were tickets to one of the major TechCrunch Disrupt conferences in San Francisco, New York, or London. Additional prizes from companies such as Amazon and Jawbone were also secured.
Taking the Initial Survey
The survey was designed by Reframe It’s team led by Dr. Alice Siu and is below in Appendix A (RFI Immigration Survey). All participants took the survey during Spring and Summer 2014.
The Deliberative Experiment Event
Briefing Document Creation
The first step in the process was the creation of the Briefing Document. To construct this document, the briefing committee convened multiple meetings via Google Hangout, communicated in smaller sub-groups through email and phone conversations in order to develop a balanced approach to the research done by the research team at Reframe It.
After an initial sign-off, the document was placed online for the tech community to comment.
2000 participants made more than 300 comments online. Those comments were then incorporated in the document. More than 70 revisions to the briefing document were the result.
In-person Deliberation Experiment
Four events were held: on August 23rd and 24th and on September 5th and 6th. Participants let selected dates provided by the Reframe It team know their availability when completing the initial survey.
- Participants were randomly assigned into control or treatment
- Moderators were selected and trained
- Those who had been selected for the treatment group were randomly assigned to a small group and moderator.
- Google Hangouts were technologically limited to 10 seats. Therefore, the moderators’ groups could be no more than 8 participants as one space was for the Reframe It recorder, and one space was for the moderator. After an initial grace period of roughly 30 minutes allowing everyone to check-in, some groups were consolidated in order to enliven the discussion, aiming for at least 5 participants per group (in addition to the moderator and the Reframe It recorder).
The schedule was 90-110 minutes of small group discussion of the Briefing Document followed by 55-80 minutes of a plenary session, a lunch break, and then another small group discussion of similar length followed by another plenary session. Each plenary session was structured around allowing for each small group to have their small groups’ questions answered. Discussions were structured around the briefing materials. The trained moderators ensured that every voice was heard and that no voice dominated the discussion.
The plenary session on August 23rd was moderated by Dr. Alice Siu. Plenary sessions on August 24th, September 5th and 6th were moderated by Bobby Fishkin. In every plenary, moderators read to the experts questions that had been posed and framed by small groups. These specific questions were answered by experts and the experts responded to each other’s comments.. Depending on the topic, experts would sometimes disagree strongly with one another, and sometimes find agreement on one or another point. The experts were not pushed for consensus by the moderator, but were only asked to respond to what they thought was important in the question and in their fellow plenary experts’ remarks. The experts did not know who raised the questions. After the second plenary session, a second survey was distributed to the participants.
Below are selected plenary questions asked by the small groups:
“Why is it the case and how has it happened that before any immigration reform can occur, politicians are requiring increased border protection (e.g., national guard troops, more agents, more fencing), given the high current density of agents (>10 per mile) and the dramatic decrease in illegal border crossings in the last few years?”
“What deterrents do other countries use for border protection that the US doesn’t currently use but could consider?”
“Why isn’t the e-verify system used nationwide?”
“How can we promote the health of the community if unauthorized immigrants are excluded from preventative services (such as Medicare) and law enforcement protection for fear of deportation?”
“So much of our debate over immigration policy and enforcement comes down to questions of shared values and their relative priority. Separate from specifics of policy, is anyone at the national level taking on helping citizens and politicians hear from one another about values? For example: fairness, openness, equality, universal education.“
“There are claims that President Obama has deported more undocumented immigrants than his predecessors yet he has promised prosecutorial discretion in trying to deport immigrants with criminal backgrounds. What is the real situation on this?“
“Are there enough jobs in the US to support the number of illegal immigrants here if they were given citizenship?
“Is it possible to raise the green card cap to lower the wait time for certain countries? Would there have been an alternate immigration policy that could have eliminated or prevented some of the backlog? Would it be possible within our current immigration system to reduce or eliminate backlogs? “
“Would it be possible to provide legal assistance to unauthorized or undocumented immigrants? Could this reduce deportations? “
“Can anyone prove that an immigrant has come to the US purely to benefit from our social services? Can immigrants benefit from the US social services in place? Can they obtain services if they don’t pay into the system, other than schools and emergency rooms?“
(3) Data
Please see the Appendix for detailed data and results
- “Reframe It Survey Results–Treatment Significant Only” describes only those results that are statistically significant. These numbers represent the “before and after” of the treatment group — the group that participated in the discussions and plenaries.
- “Reframe It Participant Demographics” data describes the attributes of the control group vs. the participants. The control group and the participants were broadly similar in attitude and in demographics in almost all categories. The only significant difference was that the control group was more likely to be employed, and less likely to be looking for work.
- “Reframe It Knowledge” pulls out those questions in the survey that are related specifically to understanding relevant facts. (NOTE: Of these questions, only three: #37, 39, and 42 are statistically significant.)
(4) Conclusions
The experiment was constructed in order to gain some quantitative measure as to how technology community members might change their opinions about immigration public policy, immigrant issues and immigration reforms if they were able to fully understand facts, assumptions, and trade-offs. After a day of dialogue and an opportunity to read briefing materials, technology community members demonstrably changed their attitudes in statistically significant ways on multiple issues.
The controlled experiment shows changes in attitude about immigration policies that are not easily mapped onto a left-right political divide. The percentage of participants who felt it would be desirable to “Eliminate the employment-based green card backlog and exempt dependents” increased from 50% to 68%, while opposition to it fell from 32% to 17%. Meanwhile, the percentage of those who felt that would be desirable to “Create a limit of H1B visas per company as a percentage of their US-based workforce” increased from 50% to 73% while the percentage who felt it undesirable fell from 27% to 12%. The percentage of respondents who felt that it was desirable for “Increase in salaries for H1B visas to be tied to local median wage for a given occupation” grew from 46% to 72% while the percentage who felt it undesirable fell from 31% to 16%. The percentage of respondents who thought it was a good idea to “Increase the total number of green cards issued annually by 100%” increased from 48% to 65% while the percentage that felt it was undesirable decreased from 36% to 25%.
Further, the percentage of deliberative participants who felt that the issuing of work visas should take into account what skills people have increased from 52% to 60%.
The changes reflect a nuanced understanding of the goals of immigration. There was an increase in support for giving priority to noncitizen parents of US citizen children for immigration, but support for automatically allowing siblings the right to permanent residency decreased. The percentage of participants who supported “Do not automatically allow the siblings of all American citizens to immigrate to the US as permanent residents” rose from 43% to 54%, while those who thought it was undesirable fell from 43% to 33%. While support for sibling immigration visas declined, support for legalizing the status of DREAMers, those who grew up in the U.S. as children but are unauthorized immigrants, increased. Support for “Provide DREAMers with a faster path to work authorization” increased from 61% to 81% while opposition fell from 31% to 20%. For participants, the desirability of “Give priority to noncitizen parents of US citizen children in the visa application process” increased from 44% to 64% after discussion.
Respondents who “strongly opposed” “Reinforcing border control” increased from 11% to 19% while those who “Strongly supported” reinforcing border control fell from 39% to 25%. Fewer than half of the respondents (49%) supported “somewhat” or “strongly” family reunification regardless of immigration status before discussion, while more than half (56%) supported it after discussion. The percentage who “strongly supported” allowing more families to be reunited through immigration rose from 33% to 41%, while the percentage of respondents who strongly opposed more families being reunited through immigration fell by more than half, from 7% to less than 3%. The percentage of respondents who felt that working immigrants are likely to contribute to our pension system rose from 29% to 36%. Also striking was the response to this question about whether immigrants threatened or did not threaten American culture: “On a scale from 0 to 10, where 0 means that immigrants have a lot to offer to the US’s cultural life, 10 means that immigrants threaten the US’s culture, and 5 is exactly in the middle, where would you position yourself on this scale, or haven’t you thought much about that?” On this question, 37% initially before dialogue identified US culture as under threat from immigration. After dialogue, that number declined to 20%.
There was a significant decline in the number of participants who believed that immigrants take jobs away from people who are already in the US. The question read, “On a scale from 0 to 10, where 0 means that immigrants take jobs from people who are already in the US, regardless of where they were born, 10 means that immigrants take the sorts of jobs that people already in the US don’t want, regardless of where they were born, don’t want and 5 is exactly in the middle, where would you position yourself on this scale, or haven’t you thought much about that?” The participants who felt that immigrants took away jobs from those already in the US (i.e., those who were in the bottom part of the scale, 0-4) declined from 38% to 27%. The participants who believed immigrants were not displacing US workers (i.e. those were in the 6-10 part of the scale) increased from 46% to 58%.
There were statistically significant changes in participants’ attitudes towards immigrants, as well.
- The percentage of respondents who would characterize unauthorized immigrants as “honest” went from 39% to 53%.
- The percentage of respondents who thought that immigrants were “law abiding” went from 35% to 47%.
- The percentage of respondents who thought that immigrants were respectful of women went from 36% to 47%.
- The percentage of respondents who thought that unauthorized immigrants were tolerant went from 44% to 52%.
Participants’ attitudes towards themselves changed in some cases, too: For the question, “I have opinions about politics that are worth listening to,” the percentage that agreed with the statement rose from 42% agree to 48%.
The changes in support for specific policy options that resulted from a day of dialogue, deliberation, input from experts, and nuanced consideration of the trade-offs and implications of those policies suggest a menu of immigration reforms that would be welcomed by an informed and engaged tech community.
About The Co-Sponsoring Organizations & Institutions
The Knight Foundation supports transformational ideas that promote quality journalism, advance media innovation, engage communities and foster the arts. The Knight Foundation believes that democracy thrives when people and communities are informed and engaged. The Knight Foundation believes that democracy can’t prosper without informed and engaged communities. The Knight Foundation’s mission is to help people be informed and engaged fully in the life of their communities. Since 2007, Knight has invested more than $150 million in new technologies and techniques related to media innovation.
TechCrunch is a leading technology media property, which attracts millions of unique visitors per month. Its voice on public policy issues has informed the technology community on a variety of debates about technology policy and issues that affect the technology community that are handled by Congress.
Silicon Valley Community Foundation advances innovative philanthropic solutions to challenging problems, engaging donors to make our region and world a better place for all. Silicon Valley Community Foundation is one of the largest community foundations in the world, with more than 1,600 philanthropic funds and $2.9 billion in assets under management. Silicon Valley Community Foundation partners with families, individuals and corporations to manage and facilitate their philanthropy. Silicon Valley Community Foundation is a comprehensive center of philanthropy. Through visionary leadership, strategic grantmaking and world-class experiences, Silicon Valley Community Foundation partners with donors to strengthen the common good locally and throughout the world.
The Deliberative Technology Community Fund was created for this project and is hosted by project co-sponsor and Knight Foundation fiscal agent the Miami Community Foundation (You can see the Deliberative Technology Community Fund by going here.)
About Reframe It
Reframe It is a technology and deliberative consulting company specializing in bringing a range of innovations to communities around nation and the world the world. This project is part of our deliberative portfolio. Building on Deliberative Polling, Reframe it offers a comprehensive suite of technical and social process tools for consulting medium and large populations about priorities, polices and innovations that could solve their problems. Our collective intelligence methodology won the McKInsey/Harvard Business Review Management 2.0 Challenge and has been successfully applied inside of a Fortune 1000 company. Reframe It’s methods can be used to consult the employees of a medium or large company, or to consult the citizens of a city, region, or country. Reframe It’s technology includes annotation technology allowing large numbers of people to participate in a conversation about the same document in specific ways.
About Reframe It’s Methodology
Many publications survey their readers about what they think. But these surveys often yield top-of-the-head responses rather than deep-seated beliefs. Reframe It’s board member James Fishkin invented a methodology called Deliberative Polling® to measure what a scientifically selected random sample of a community would think if they were able to deeply consider a nuanced issue. The methodology uses controlled experimental conditions to allow a random sample of readers engage in small-group discussion with trained moderators, to review materials created by diverse, disagreeing experts, and to measure the attitude of participants before and after dialogue. This approach was combined with Reframe It’s technology to win the McKinsey/Harvard Business Review management 2.0 Challenge as a collective intelligence process.
The methodology used in the TechCrunch National Online Dialogue on Immigration Reform conducted by Reframe It in collaboration with the Knight Foundation and Silicon Valley Community Foundation adapts its methodology to the circumstances of consulting the tech community. Reframe It recruited tech community members through a variety of means described above, and included in the sample tech community members from various parts of the country, of various educational and ethnic backgrounds. These individuals were then randomly assigned to treatment group and control group. The treatment group experienced conditions like a Deliberative Poll. The project methodology also adapts the process that won the McKinsey/HBR Management 2.0 Challenge. There was insufficient time for a deliberative random sample microcosm of the tech community to rate the comments, so the working group and briefing committee worked extensively to incorporate more than 75 revisions of the briefing materials based on public annotations.
Reframe It Team
Bobby Fishkin CEO, Reframe It Inc.
Dr. Alice Siu, Lead Deliberative Consultant, Reframe It Inc.
Jessica Margolin, VP Program Management, Reframe It Inc.
Ben Taitelbaum, CTO, Reframe It Inc.
Sarah Zawacki, Analyst
James Fishkin, Reframe It Board of Directors Member
David Witzel (Reframe It Advisor)
Aldo Bello (Briefing committee recruitment, plenary expert recruitment and videography for Mind Media Productions- in-kind supporter of project)
Jason Hunter- Videography from Mind Media Productions
Silicon Valley Community Foundation received approval from the Office of the Attorney General of California to conduct a raffle for 30 tickets to TechCrunch Disrupt that are “earned” by participants through a day’s volunteer time participating in the Deliberative Poll if they are in the treatment group and appropriate participation for the control group. This is the first known example of inter-state online volunteerism incentivized by Raffle approved by the CA Attorney General’s Office. In this case the volunteerism was in the form of deliberation, in which participants earned a chance to win a ticket to TechCrunch Disrupt or a piece of technology.
Acknowledgments
Special thanks to Knight Foundation for the grant that made this possible, the Miami Community Foundation for hosting the Deliberative Technology Community Fund, to the briefing committee members, plenary experts, trained moderators, and of course, the participants themselves. Special thanks also to David Witzel, Aldo Bello, all of those who took time out from their busy lives to give us advice during the creation of the briefing materials.
Appendix A: Statistically Significant Changes of Opinion
Online Deliberative Experiment
Results: Policy Changes for Participants in the Online Deliberation
Significant Changes Only
September 2014
Note: T1 denotes Before Deliberation; T2 denotes After Deliberation; T2-T1 denotes after deliberation minus before deliberation; Sig. denotes statistical significance.
In the Sig. column, * indicates significance 0.10 or below, ** for 0.05, and *** for 0.01 or below. a denotes question changed significantly for control group
The first row for each question shows the means for each question and the subsequent rows show the percentages. For the purposes of this document, the answer scales are collapsed for certain questions.
| T1 | T2 | T2-T1 | Sig. | |
| 4. And, on the same scale, generally speaking, how serious a problem or not would you say temporary work visas are? | 4.707 | 5.256 | .548 | .055* |
| 0-4 no problem at all | 48.8 | 35.7 | ||
| 5 | 12.2 | 11.9 | ||
| 6-10 most serious problem we face | 39.0 | 52.4 | ||
| (DK/NA) | (2.4) | (1.2) | ||
| 5b. Eliminate the employment-based green card backlog and exempt dependents | 5.549 | 6.563 | 1.014 | .019** |
| 0-4 undesirable | 32.4 | 16.9 | ||
| 5 | 17.6 | 15.6 | ||
| 6-10 desirable | 50.0 | 67.5 | ||
| (DK/NA) | (11.9) | (6.1) | ||
| 5e. Create a limit of H1B visas per company as a percentage of their US-based workforce | 5.608 | 6.971 | 1.362 | .001*** |
| 0-4 undesirable | 27.0 | 11.7 | ||
| 5 | 23.0 | 15.6 | ||
| 6-10 desirable | 50.0 | 72.7 | ||
| (DK/NA) | (11.9) | (8.3) | ||
| 5g. Increase salaries for H1B visas to be tied to local median wage for a given occupation | 5.625 | 7.014 | 1.389 | .000*** |
| 0-4 undesirable | 31.1 | 16.1 | ||
| 5 | 23.0 | 12.4 | ||
| 6-10 desirable | 45.9 | 71.6 | ||
| (DK/NA) | (10.8) | (2.4) | ||
| 5i. Increase EB-5’s limit to 10,000 families rather than 10,000 individuals | 5.937 | 6.762 | .825 | .025**a |
| 0-4 undesirable | 26.9 | 28.2 | ||
| 5 | 13.4 | 6.4 | ||
| 6-10 desirable | 59.7 | 65.4 | ||
| (DK/NA) | (18.3) | (6.0) | ||
| 5j. Do not automatically allow the siblings of all American citizens to immigrate to the US as permanent residents | 4.974 | 5.923 | .948 | .035** |
| 0-4 undesirable | 42.5 | 32.9 | ||
| 5 | 15.0 | 13.4 | ||
| 6-10 desirable | 42.5 | 53.7 | ||
| (DK/NA) | (4.8) | (3.5) | ||
| 5n. Increase the number of prosecutions for border crossings to up to 210 per day | 5.438 | 4.726 | -.712 | .068* |
| 0-4 undesirable | 43.4 | 45.0 | ||
| 5 | 9.21 | 10.0 | ||
| 6-10 desirable | 55.3 | 45.0 | ||
| (DK/NA) | (9.5) | (5.9) | ||
| 5q. Introduce a federal mandate on employment verification systems | 6.129 | 6.957 | .829 | .032** |
| 0-4 undesirable | 25.0 | 15.4 | ||
| 5 | 14.5 | 6.4 | ||
| 6-10 desirable | 60.5 | 78.2 | ||
| (DK/NA) | (9.5) | (4.9) | ||
| 0-4 | 31.5 | 26.6 | ||
| 5 | 17.8 | 10.1 | ||
| 6-10 | 50.7 | 63.3 | ||
| (DK/NA) | (13.1) | (7.1) | ||
| 5u. Provide DREAMers with a faster path to work authorization | 5.918 | 7.123 | 1.205 | .001*** |
| 0-4 undesirable | 30.7 | 19.5 | ||
| 5 | 8.0 | 2.4 | ||
| 6-10 desirable | 61.3 | 78.1 | ||
| (DK/NA) | (10.7) | (3.5) | ||
| 5v. Give priority to noncitizen parents of US citizen children in the visa application process | 4.913 | 6.300 | 1.388 | .000*** |
| 0-4 undesirable | 43.2 | 26.2 | ||
| 5 | 12.4 | 9.5 | ||
| 6-10 desirable | 44.4 | 64.3 | ||
| (DK/NA) | (3.6) | (1.2) | ||
| 5w. Have applicants return to home country while waiting for visa | 4.553 | 4.066 | -.487 | .099* |
| 0-4 undesirable | 56.3 | 56.3 | ||
| 5 | 8.8 | 10.0 | ||
| 6-10 desirable | 35.0 | 33.8 | ||
| (DK/NA) | (4.8) | (5.9) | ||
| 5x. Increase the total number of green cards issued annually by 100% | 5.420 | 6.159 | .739 | .036**a |
| 0-4 undesirable | 35.6 | 25.3 | ||
| 5 | 16.4 | 10.1 | ||
| 6-10 desirable | 48.0 | 64.6 | ||
| (DK/NA) | (13.1) | (6.0) | ||
| 8. Some people think that the US should send all illegal immigrants back to the countries they came from. Suppose these people are at one end of a 0-to-10 scale, at point
0. Other people think that the US should legalize all the illegal immigrants currently here. Suppose these people are at the other end of the scale, at point 10. People who are exactly in the middle are at point 5, and of course other people have opinions at other points between 0 and 10. Where would you place your views on this scale, or haven’t you thought much about that? |
4.880 | 5.711 | .831 | .002*** |
| 0-4 | 41.7 | 27.4 | ||
| 5 | 10.7 | 11.9 | ||
| 6-10 | 47.6 | 60.7 | ||
| (DK/NA) | (3.6) | (1.2) | ||
| Strongly agree (0) | 14.6 | 19.5 | ||
| Somewhat agree | 26.8 | 31.7 | ||
| Neither agree nor disagree | 14.6 | 11.0 | ||
| Somewhat disagree | 22.0 | 24.4 | ||
| Strongly disagree (1) | 22.0 | 13.4 | ||
| (DK/NA) | (2.4) | (3.5) | ||
| 10c. Decisions about what WORK VISAS to issue should take into account what job skills they have. | .216 | .153 | -.063 | .028** |
| Strongly agree (0) | 52.4 | 60.2 | ||
| Somewhat agree | 30.5 | 25.3 | ||
| Neither agree nor disagree | 4.9 | 8.4 | ||
| Somewhat disagree | 4.9 | 2.4 | ||
| Strongly disagree (1) | 7.3 | 3.6 | ||
| (DK/NA) | (2.4) | (2.4) | ||
| 0-4 | 12.1 | 9.4 | ||
| 5 | 9.6 | 7.1 | ||
| 6-10 | 78.3 | 83.5 | ||
| (DK/NA) | (1.2) | (0.0) | ||
| 11b. Having close family in the US | 5.317 | 6.329 | 1.012 | .003*** |
| 0-4 unimportant | 33.7 | 20.2 | ||
| 5 | 14.5 | 11.9 | ||
| 6-10 important | 51.8 | 67.9 | ||
| (DK/NA) | (1.2) | (1.2) | ||
| 11d. Being able to support oneself financially | 7.614 | 8.084 | .470 | .084* |
| 0-4 unimportant | 9.6 | 5.9 | ||
| 5 | 12.1 | 7.1 | ||
| 6-10 important | 78.3 | 87.1 | ||
| (DK/NA) | (1.2) | (0.0) | ||
| 11e. Commitment to the US’s way of life | 6.723 | 6.940 | .217 | .514 |
| 0-4 unimportant | 21.7 | 17.7 | ||
| 5 | 10.8 | 10.6 | ||
| 6-10 important | 67.5 | 71.8 | ||
| (DK/NA) | (1.2) | (0.0) | ||
| 12. And how strongly would you favor or oppose each of the following?
a. Reinforcing border controls |
.649 | .542 | -.107 | .000*** |
| Strongly oppose (0) | 10.7 | 18.8 | ||
| Somewhat oppose | 16.7 | 16.5 | ||
| Neither support nor oppose | 14.3 | 18.8 | ||
| Somewhat support | 19.1 | 21.2 | ||
| Strongly support (1) | 39.3 | 24.7 | ||
| (DK/NA) | (0.0) | (0.0) | ||
| c. Allowing more families to be reunited regardless of immigration status | .554 | .635 | .080 | .017** |
| Strongly oppose (0) | 14.5 | 7.6 | ||
| Somewhat oppose | 19.3 | 16.5 | ||
| Neither support nor oppose | 16.9 | 20.3 | ||
| Somewhat support | 28.9 | 26.6 | ||
| Strongly support (1) | 20.5 | 29.1 | ||
| (DK/NA) | (1.2) | (6.0) | ||
| d. Allowing more families to be reunited for legal immigrants | .708 | .772 | .064 | .054** |
| Strongly oppose (0) | 7.3 | 2.5 | ||
| Somewhat oppose | 4.9 | 5.0 | ||
| Neither support nor oppose | 15.9 | 15.0 | ||
| Somewhat support | 39.0 | 36.3 | ||
| Strongly support (1) | 32.9 | 41.3 | ||
| (DK/NA) | (2.4) | (2.4) | ||
| 16. How strongly would you agree or disagree with each of the following statements?
a. The contributions from working immigrants will help maintain the pension system. |
.350 | .297 | -.053 | .066* |
| Strongly agree (0) | 29.0 | 36.1 | ||
| Somewhat agree | 32.9 | 32.5 | ||
| Neither agree nor disagree | 15.8 | 15.7 | ||
| Somewhat disagree | 11.8 | 6.0 | ||
| Strongly disagree (1) | 10.5 | 9.6 | ||
| (DK/NA) | (8.4) | (2.4) | ||
| 18. On a scale from 0 to 10, where 0 means that immigrants have a lot to offer to the US’s cultural life, 10 means that immigrants threaten the US’s culture, and 5 is exactly in the middle, where would you position yourself on this scale, or haven’t you thought much about that? | 3.602 | 2.928 | -.675 | .026** |
| 0-4 | 61.9 | 67.9 | ||
| 5 | 10.7 | 11.9 | ||
| 6-10 | 37.4 | 20.2 | ||
| (DK/NA) | (0.0) | (1.2) | ||
| 19. On a scale from 0 to 10, where 0 means that immigrants take jobs from US-born, 10 means that immigrants take the sorts of jobs that US-born don’t want and 5 is exactly in the middle, where would you position yourself on this scale, or haven’t you thought much about that? | 5.524 | 5.988 | .463 | .097* |
| 0-4 | 29.3 | 25.9 | ||
| 5 | 19.5 | 17.7 | ||
| 6-10 | 51.2 | 56.5 | ||
| (DK/NA) | (2.4) | (0.0) | ||
| 20. On a scale from 0 to 10, where 0 means that immigrants take jobs from people who are already in the US, regardless of where they were born, 10 means that immigrants take the sorts of jobs that people already in the US don’t want, regardless of where they were born, don’t want and 5 is exactly in the middle, where would you position yourself on this scale, or haven’t you thought much about that? | 5.295 | 5.859 | .564 | .048** |
| 0-4 | 37.5 | 26.5 | ||
| 5 | 16.3 | 15.7 | ||
| 6-10 | 46.3 | 57.8 | ||
| (DK/NA) | (4.8) | (2.4) | ||
| 21. Of all the immigrants in the US, what percentage do you think are here with no legal status? Please answer with a number between 0 and 100. If you don’t know, you may write don’t know and move on to the next question. | 37.3 | 30.3 | -6.93 | .012** |
| 25. On a scale from 0 to 10, where ‘0’ is “not at all”, ’10’ is “completely”, and ‘5’ is “exactly in the middle”, how much would you say each of the following describes people with WORK visas living in the US? | ||||
| d. Respectful of women | 5.938 | 6.609 | .672 | .009*** |
| 0-4 | 20.6 | 11.3 | ||
| 5 | 27.9 | 18.3 | ||
| 6-10 | 51.5 | 70.4 | ||
| (DK/NA) | (19.1) | (16.5) | ||
| 26. On a scale from 0 to 10, where ‘0’ is “not at all”, ’10’ is “completely”, and ‘5’ is “exactly in the middle”, how much would you say each of the following describes immigrants with no legal status living here in the US?
a. Honest |
4.878 | 5.689 | .811 | .006*** |
| 0-4 | 41.6 | 33.3 | ||
| 5 | 19.5 | 14.1 | ||
| 6-10 | 39.0 | 52.6 | ||
| (DK/NA) | (8.3) | (8.2) | ||
| c. Law-abiding | 4.685 | 5.411 | .726 | .013** |
| 0-4 | 48.7 | 40.3 | ||
| 5 | 16.7 | 13.0 | ||
| 6-10 | 34.6 | 46.8 | ||
| (DK/NA) | (6.0) | (9.4) | ||
| d. Respectful of women | 5.095 | 5.762 | .667 | .009*** |
| 0-4 | 34.3 | 25.0 | ||
| 5 | 30.0 | 27.9 | ||
| 6-10 | 35.7 | 47.1 | ||
| (DK/NA) | (16.7) | (19.1) | ||
| e. Tolerant | 5.355 | 5.871 | .516 | .058* |
| 0-4 | 27.8 | 25.0 | ||
| 5 | 27.8 | 23.5 | ||
| 6-10 | 44.4 | 51.5 | ||
| (DK/NA) | (14.3) | (18.1) | ||
| 28. On a 0 to 10 scale, where 0 is “not at all”, 10 is “as much as could reasonably be expected,” and 5 is exactly in the middle, how able would you say Congress is able to get important things done? | 2.904 | 2.313 | -.590 | .018** |
| 0-4 | 72.3 | 86.9 | ||
| 5 | 6.0 | 2.4 | ||
| 6-10 | 21.7 | 10.7 | ||
| (DK/NA) | (1.2) | (1.2) | ||
| 31. Here are some things that people find more or less important for themselves or society to have. On a 0 to 10 scale, where 0 is extremely unimportant, 10 is extremely important, and 5 is exactly in the middle, how important or unimportant would you say each of the following is to you? | ||||
| b. Leaving people and companies free to compete economically | 6.812 | 7.207 | .390 | .056** |
| 0-4 | 22.0 | 15.3 | ||
| 5 | 9.8 | 8.2 | ||
| 6-10 | 68.3 | 76.5 | ||
| (DK/NA) | (2.4) | (0.0) | ||
| g. Getting to decide exactly what to do with everything I earn | 6.530 | 6.964 | .434 | .075* |
| 0-4 | 25.3 | 11.8 | ||
| 5 | 6.0 | 14.1 | ||
| 6-10 | 68.7 | 74.1 | ||
| (DK/NA) | (1.2) | (0.0) | ||
| m. Having a safe community | 8.952 | 8.663 | -.289 | .056* |
| 0-4 | 1.2 | 2.4 | ||
| 5 | 3.6 | 2.4 | ||
| 6-10 | 95.2 | 95.3 | ||
| (DK/NA) | (1.2) | (0.0) | ||
| 32b. Most public policy issues are so complicated that a person like me can’t really understand what’s going on. | .705 | .610 | -.095 | .004*** |
| Strongly agree (0) | 2.4 | 9.4 | ||
| Somewhat agree | 19.1 | 22.4 | ||
| Neither agree nor disagree | 11.9 | 11.8 | ||
| Somewhat disagree | 27.4 | 25.9 | ||
| Strongly disagree (1) | 39.3 | 30.6 | ||
| (DK/NA) | (0.0) | (0.0) | ||
| 32d. I have opinions about politics that are worth listening to. | .205 | .199 | -.006 | .085* |
| Strongly agree (0) | 42.2 | 48.2 | ||
| Somewhat agree | 45.8 | 30.6 | ||
| Neither agree nor disagree | 4.8 | 15.3 | ||
| Somewhat disagree | 2.4 | 3.5 | ||
| Strongly disagree (1) | 4.8 | 2.4 | ||
| (DK/NA) | (1.2) | (0.0) | ||
| 33d. They have good reasons; there are just better ones on the other side. | .399 | .332 | -.067 | .042** |
| Strongly agree (0) | 14.6 | 15.3 | ||
| Somewhat agree | 36.6 | 50.6 | ||
| Neither agree nor disagree | 31.7 | 20.0 | ||
| Somewhat disagree | 8.5 | 11.8 | ||
| Strongly disagree (1) | 8.5 | 2.4 | ||
| (DK/NA) | (2.4) | (0.0) |
[1] Offering cash incentives and raffle tickets in exchange for participation prevents, say, only those who feel strongly one way or the other about immigration to participate.
[2] Participants in the first weekend were told they would receive $50, while those in the second weekend were told they would receive $100. If participants received the grand prize if they would not receive cash compensation.
Reframe It 