Quick answer
In Luo et al.’s outbound loan-renewal field experiment, upfront AI identification was associated with a reported 56.3% immediate hang-up rate and a 4.8% renewal rate. The same automated agent’s no-disclosure arm reported 23.7% renewal. These are historical phone results and should not be used as ecommerce conversion forecasts.
Key findings and sources
Upfront and no-disclosure renewal differed by 18.9 points
Offload Rep’s reading of Luo et al. Table 3 finds a 4.8% loan-renewal rate in the upfront AI identification arm and 23.7% in the no-disclosure arm, an 18.9-point difference in an outbound financial-services experiment.
Six randomized conditions in an outbound loan-renewal promotion conducted by a Chinese financial services company; Marketing Science, 2019; historical field experiment. Convert the Table 3 purchase-rate decimals to percentages and subtract.
The immediate hang-up difference was 51.8 points
Luo et al. reported immediate hang-up rates of 56.30% for disclosure before conversation and 4.50% for disclosure after conversation. The reported difference is 51.80 percentage points.
Six randomized conditions in an outbound loan-renewal promotion conducted by a Chinese financial services company; Marketing Science, 2019; historical field experiment. Subtract the two published arm-level hang-up rates; see Figure 1 for the paper’s immediate-exit definition.
Call length differed by 53.827 seconds
In Luo et al.’s historical phone experiment, reported call length was 10.325 seconds with upfront AI identification and 64.152 seconds when identity was not disclosed, a difference of 53.827 seconds.
Six randomized conditions in an outbound loan-renewal promotion conducted by a Chinese financial services company; Marketing Science, 2019; historical field experiment. Subtract published mean call lengths in Table 3.
The arm with higher initial response had lower renewal
Luo et al.’s upfront disclosure and no-disclosure arms reported response rates of 96.52% and 95.79%, respectively. Their renewal rates ran in the other direction, 4.8% versus 23.7%.
Six randomized conditions in an outbound loan-renewal promotion conducted by a Chinese financial services company; Marketing Science, 2019; historical field experiment. Compare the separate response-rate and purchase-rate columns without treating them as the same event.
Compare the results
Conversation exit and final decision answer different questions
Record first-stage exits separately from final decisions.
Treat initial response and conversation continuation as separate outcomes
A customer who opens a support chat can still stop after the first reply. In Luo and colleagues’ six-arm phone experiment, the upfront-disclosure group had a 96.52% call response rate. It also had a reported 56.30% immediate hang-up rate. A report that shows only initial response would miss the second outcome entirely.
For store support, count the chance to see the automated greeting and the chance to continue after that greeting separately. If the first useful reply explains how to reach a human and what can actually be checked, test that design while clearly identifying the agent.
Read the measured decision in its own setting
The original purchase outcome was an agreement to renew a loan, following a scripted outbound promotion. The reported 4.8% versus 23.7% comparison belongs to that decision, setting and population. A shopper asking for help with a shipment is neither making the same decision nor initiating the same kind of interaction.
The transferable lesson is measurement design. Ask whether the first reply made the scope clear, whether the customer stayed for a substantive exchange, and whether the actual issue was resolved. A drop in one event can change who remains available to answer a follow-up survey.
Compare designs that keep identity clear
An ecommerce team can hold transparent bot identification constant while testing the specificity of the first useful message and the visibility of the human option. For a shopper stuck applying a discount, identify what was checked and what the next person can see. Give a human a short record of what failed rather than asking the shopper to restart.
Include early exits in the assigned cohort. If repeat-contact rates are only measured among customers who reach a human, a change that pushes frustrated customers out may appear to improve the reported support result.
Study data
| Measure | Value |
|---|---|
| Less experienced human attempted calls | 1,053 calls |
| Less experienced human response rate | 94.96% |
| Less experienced human hang-up rate | 0% |
| Less experienced human call length | 39.888 seconds |
| Less experienced human loan renewal rate | 4.9% |
| Proficient human attempted calls | 1,042 calls |
| Proficient human response rate | 95.97% |
| Proficient human hang-up rate | 0% |
| Proficient human call length | 63.888 seconds |
| Proficient human loan renewal rate | 25.1% |
| AI identity not disclosed attempted calls | 1,044 calls |
| AI identity not disclosed response rate | 95.79% |
| AI identity not disclosed hang-up rate | 0% |
| AI identity not disclosed call length | 64.152 seconds |
| AI identity not disclosed loan renewal rate | 23.7% |
| AI identity disclosed before conversation attempted calls | 1,036 calls |
| AI identity disclosed before conversation response rate | 96.52% |
| AI identity disclosed before conversation hang-up rate | 56.3% |
| AI identity disclosed before conversation call length | 10.325 seconds |
| AI identity disclosed before conversation loan renewal rate | 4.8% |
| AI identity disclosed after conversation attempted calls | 1,044 calls |
| AI identity disclosed after conversation response rate | 95.78% |
| AI identity disclosed after conversation hang-up rate | 4.5% |
| AI identity disclosed after conversation call length | 63.873 seconds |
| AI identity disclosed after conversation loan renewal rate | 11.0% |
| AI identity disclosed after decision attempted calls | 1,036 calls |
| AI identity disclosed after decision response rate | 96.52% |
| AI identity disclosed after decision hang-up rate | 0% |
| AI identity disclosed after decision call length | 63.731 seconds |
| AI identity disclosed after decision loan renewal rate | 23.2% |
What changes when you compare the figures
| Comparison | Result | Calculation |
|---|---|---|
| Renewal gap, undisclosed minus upfront | 18.9 percentage points | = 23.7 - 4.8 |
| Hang-up gap, upfront minus after conversation | 51.8 percentage points | = 56.30 - 4.50 |
| Call duration gap, no disclosure minus upfront | 53.827 seconds | = 64.152 - 10.325 |
| Proficient and less experienced human renewal gap | 20.2 percentage points | = 25.1 - 4.9 |
| Upfront answer-rate lead versus no disclosure | 0.73 percentage points | = 96.52 - 95.79 |
Methodology and sources
We transcribed the six arms of Luo et al. Table 3 and converted its purchase-rate decimals to percentages. Differences are model-free contrasts of published rounded rates and call lengths. The paper’s outcome was agreeing to renew a loan. An early hang-up meant an answered call that ended within five seconds after disclosure.
Download the source data (CSV)
Download the calculations (CSV)
Download the citation notes (CSV)
Related reading
Human-Like Chatbot Conversation Statistics
AI customer support escalation statistics
AI customer support productivity statistics
Sources and definitions
The source data includes the population, period, denominator and limitations for every figure.